�� � ��� � � �� � � �� � � � � � �� �� � � � �� � �� � � � �� � � � �� � � � � ���� � � � � � � � �� � �� � � � �� � � �� � � �� � �� � � � �� � � � � � �� � � � � � � � � � � � � � � � � � � � �
� �� � � !"# ! "
$ %& '( ) *+ & ,
- . . /10 2 23 3 34 5 678 94 : 7 2<; =1>? 7@ ? > 7
( (A BC D DE
� ��� � ��� ��
� � � � �� � � � � � �� �� � �
& � � ! � � " �� � � ��� " � ! �# � �� � ! �# � � � ) � � � � � ! �� � " �� � !# � " � !
� � �� � �� �� � � �� �
� �! � " �� � �� #�$ %� � &� $ #� � $ �' � �( � �� )� ! �
� γ* Ψ* � �( � $ ! ��$ !
� + � ( ( � �� ,� � � � * , � � "� � & �$ �� � " �! �
,- . # � � � ) � � � � � ! �
� /� ( � " "� �� ' �$ # � ��$ ! � �� " �! � ) "� !
� 0 � � �! ( � �1 "� ! � � �$ # � �� �2
& � � ! � � � " �� � � � ! � �3 � ! � � �� � � !# � �� � ! ��� " � ! � ,- . # � � � ) � � � � � ! �
� � � �$ � ! � � "� � �( � $ ! ��$ �� 4 � �� � � 5� $ 6( � � &�� 7 � � " �! � � ��$ �98 � $ � ( � :� $ $ � �� � �� ( � # � � �
� � � �$ � ! ! � � "8 � � � � � � �� %� " � �� � ��$ #� � �! �� ; "� � %� * �$ � * � � � ; �� ! ,< = ( � " � �* # "� ! ! � !
( � � � � � � � �� � � � � � " 3 � � � � �� � � ! � � � �" � � " ! � ! �� � � � �" ! � ! � � " � � 3 �� ! �
� 7$ � � � ) "1 ( � #� $ �� � " � $ ) �� "� & �� ! �� � # �� � � "�
� = * ,< = � � � � �� 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� �
> � %� � "? @ @A C BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � � �� � � � � � � �� � �� �� �
* � " � � � 3 " � � � � �� � � � " � � �� # !
� � �� �� � � " �� $ � ! ! � # �� $ � �� ! � ) 5� �! x ∈ X 6 "� � � ! #� � � & � � �� ! y ∈ Y = {1, . . . , Q}
� � : � � �� 1 ! � ��2 �! �� $ #� �98 � $ #� � � "� � " �� �� �� � (X,Y ) 6 %� "� � � ! �� $ ! X × Y � �! �� � ) � � ! � � %� $ � � $ �
( � ! � � � �� � � � )� ) � " � � � P
� � � ) "1 ( � � "� "� � 5� �$ �� �� (X,Y )� ! � �$ #�$ $ � �
* ! �� � � � � � � � ��� � !
� Dm = ((Xi, Yi))1≤i≤m � m* � # � � $ � � " "�$ #�$ ! � � �� � �� #� � �� ! �� (X,Y ) �$ � � ��� $ �� $ �� !
� G � '� ( � " "� �� ' �$ # � ��$ ! g �� X �� $ ! RQ � F � '� ( � " "� �� � 1 & "� ! �� � � # �! ��$ f �� X �� $ ! Y ⋃ {∗} �
f(x) = argmax1≤k≤Q gk(x)� � f(x) = ∗ � $ #� ! �98 �2 ��� �
* ! ! ���� � �� ! " �� ! � � � �" !
� ` ' �$ # � ��$ �� �� � �� � ` (y, g(x)) = 1l{gy(x)≤maxk 6=y gk(x)} � ` (y, f(x)) = 1l{f(x)6=y} �� , � "� # � ��$ �98 � $ � ' �$ # � ��$ g∗( �$ �( �! � $ � ! � � G "� � �! �� �
R(g) = E [` (Y, g (X))] = P (f(X) 6= Y )
> � %� � "? @ @A � BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � �� � � ���� � � � � � � � � � � �� � � � � � �
B 3� � � � �� � E �A � � � � �� � M � �� �� M �� � � � �� � RQ × {1, . . . , Q} � � � R �� � � � �� �
∀(v, k) ∈ RQ × {1, . . . , Q} , M(v, k) =
1
2
(
vk − maxl 6=k
vl
)
M(v, ·) = max1≤k≤Q M(v, k)
B 3� � � � �� � C � . � " � !# � � � ) � � � � � ! � ! g � " ��� !� !# � � ! (x, y) �
∀(g, x, y) ∈ G × X × Y , M(g, x, y) = M (g(x), y)
B 3� � � � �� � � � % � 3 " � � ! " � ∆ ! � ∆∗ � g = (gk)1≤k≤Q � � � � �� � X � � � RQ
� � � � � � �� ∆g = (∆gk)1≤k≤Q� � X � � � RQ� � � �� � � � � �
∀x ∈ X , ∆g(x) = (M (g(x), k))1≤k≤Q
� � � � � � �� ∆∗g = (∆∗gk)1≤k≤Q� � X � � � RQ� � � �� � � � � �
∀x ∈ X , ∆∗g(x) = (max {∆gk(x),−M (g(x), ·)})1≤k≤Q
> � %� � "? @ @A C BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � �� � � ���� � � � � � � � � � � �� � � � � � �
∆#� � ( � "� #� ∆� � ∆∗ �� $ ! "� ! �2 � � � ! ! ��$ ! %� "� ) "� ! � � � � "� ! �� �2 � � �� � �� � � !
��2 � � R(g) = E[
1l{∆#gY (X)≤0}
]
�
B 3� � � � �� � C �& � � ! � # � " � ! � �� �� γ ∈ R∗+ � � � �! �� � 6( � � &� γ � g� �� � �� � � � � �
Rγ(g) = E[
1l{∆#gY (X)<γ}
]
=
∫
X×Y
1l{∆#gy(x)<γ}dP (x, y)
�! �� � � ( � �� � �� � 6( � � &� γ �
Rγ,m(g) =1
m
m∑
i=1
1l{∆#gYi(Xi)<γ}
A �# � � � ! � ! �� � � � �� � �� 3 � � � !" � ∆#γ G
� � � γ ∈ R∗+ ! � � � πγ : R → [−γ, γ] "� ' �$ # � ��$ " �$ �� �� � ! � �� � �� � �� $ �� �� �
πγ(t) =! � &$ � (t) · min (|t| , γ)
∆#γ g =
(
∆#γ gk
)
1≤k≤Q, ∆#
γ gk = πγ ◦ ∆#gk, ∆#γ G =
{
∆#γ g : g ∈ G
}
> � %� � "? @ @A � BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � �� � � � � � � � � � � � � ∆#γ G � � �� � � � � � � �� � � �� �
�� �� � � ε� � � � �� � � ε� � � � � �� � �98 � $ ! � � ! * � $ ! � ( ) "� E′ �98 � $ � ! �� #� � ! � � � � *( � �� � ��� � (E, ρ)
B 3� � � � �� � � �+ � # � " ! � � ! �� !" � " ! �
N (ε, E′, ρ) � � � �� � � � � � � � �� � � � � � � � � � � � �� ε� � �� �� � � � � � � �� E′ �� � +∞ �
N (p)(ε, E′, ρ) � � � ε� � � � �� � � � � � �� � � � � � � � � � � � �� � � � � E′ � � � � � � � ! � E′ �
> � %� � "? @ @A � BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� �� � �� � � � � � � � � � � � � � � �� � � � � � � � � � �
�� � � � � � � � � � � � �� � � � � �� � �� � � � � � � � �
� � 3 � " � # !E �& � � ! � � " �� � � � - � � � �� �E � �E � �� �� F� � � � � � � � � � �� � � � � � �� � � �
X � � � � � � � � � �� � � �� � � N(
F , (Xi)1≤i≤n
)
� � � �� � � � � �� � � � ��� � � � � � � � � � � �
� � � � � � � � � � �� � � � � � � (Xi)1≤i≤n � δ ∈ ]0, 1[ �� � � �� � � � � � � �� � �� � � � � � � � � � 1 − δ� �
� �� � � � � � � � � � �� f � F � � �� � � � � � � � � � � � � � �� � � �� � � � � � � �
R(f) ≤ Rm(f) +
√
1
m
(
ln(
EN(
F , (Xi)1≤i≤2m
))
+ ln
(
4
δ
))
+1
m
ln(
EN(
F , (Xi)1≤i≤2m
))
� ! � "8 �� � � � � � � �� �� F ! � � "8 � # � � $ � � " "�$ (Xi)1≤i≤2m �
> � %� � "? @ @A BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� �� � �� � � � � � � � � � � � � � � �� � � � � � � � � � �
�� � � � � � � � � � � � �� � � � � �� � �� � � � � � RQ
B 3� � � � �� � � ��� � ! �� )# 3 �" � ! dxn � � � � � �� � � � � �� xn = (xi)1≤i≤n ∈ Xn� � �� � ��
�� � � � �� � � � �� � � dxn� � � G � �� � � � �� � � � � � �
∀(g, g′) ∈ G2, dxn(g, g′) = max1≤i≤n
‖g(xi) − g′(xi)‖∞
, � � � N (ε,G, n) = supxn∈Xn N (ε,G, dxn)
� � 3 � " � # !C �& � � ! � � " �� � � � � � �� G �� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � �
RQ � � � � � � � � �� � � � �� � � � � � � � Q� � � � �� � � � � �� � � � �� � � � � � � Γ ∈ R
∗+ � δ ∈ ]0, 1[ �� �
� �� � � � � � � �� � �� � � � � � � � � � 1− δ� � � � � � � � � �� � � � � � � � � � � � γ � � � ]0,Γ]� � � �� � �
� � � � � � � �� g � G � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �
R(g) ≤ Rγ,m(g) +
√
2
m
(
ln(
2N (p)(
γ/4,∆#γ G, 2m
))
+ ln
(
2Γ
γδ
))
+1
m
> � %� � "? @ @A E BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� �� � � � �� � � �
B 3� � � � �� � � Ψ ) � �# !� � �� � � ��� !� ) B � � � �� �� � �E � � � � �� �� F� � � � � � � � � � �� � � � � �
�� � � � X � � � � � � � � � ��� � � � � � � {1, . . . , Q} � �� �� Ψ� � � � � � � � � � � � �� � � �� � ψ �
{1, . . . , Q} � � � {−1, 1, ∗}� � � � � � �� � ∗� �� � � � � � � � � �� �� � � � �� �� � �
� � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X � � � �� � �� Ψ* � � " % �� �! � � �� F� � � � � �� � �
� � � � �� � � �� ψn =(
ψ(i))
1≤i≤n
� � � Ψn � � � � � �� � � � � � � � � � � vy � {−1, 1}n� � � � �� � �
� � � �� fy � � � F� � � �� � �� � �
(
ψ(i) ◦ fy(xi))
1≤i≤n= vy
� � Ψ* � �( � $ ! ��$ � F� � � � Ψ� � � � (F)� � � � � � � � � � � �� � �� � �� � � � � � � � � � � � � XΨ� �� � � � �� � � �� F� � � � � � � � � � � � � � �� � �� � � � � � � � � � � � � �� � �� �
& !# � " !E �� � � F � Ψ � � � � � � � � � � � � �� � �� � � � � �� � � � � � � � � � � � � �� �� � �� ��
� �� � � � � �� � � � � � � � � � � � � �� � � � � � � � � � � � � �� � � � � � � � � � � � �� � �� � � � � � �
� � � � � � � � {−1, 1, ∗}� � � � �� � � � � �� � � � � � � � �� �� � � �� � �� � � � � � �� � � �� � � � � �
� � � � � � � �� � �
Ψ� � � � (F) = � � � � � ({(x, ψ) 7→ ψ ◦ f(x) : f ∈ F , ψ ∈ Ψ})
> � %� � "? @ @A � BDCE
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � � � � � � � � �� �� � � � � � �� � � � �� � � �
B 3� � � � �� � E � B �# !� � �� � � " � � � � ! � B � � ! � �E �E �+ � � � " � � �� �E �E � � �� �� F� � � � � �
� � � � �� � � � � � �� � � � X � � � � � � � � � {1, . . . , Q} � � � � �( � $ ! ��$ &� � � � � �� � � F�
� � � � � (F)� � � �� Ψ� � � � � �� � F � � � � � � � � � � � �� � � � � � Ψ = {ψk : 1 ≤ k ≤ Q}� �� � � � � �� � � ��
ψk �� � � �� � � � � 1� � � � � � �� � � � � � � � � � k � �� � � � � −1 � � � � � � � � � �� � �� � �
� � � � � �� � � � � �� � �� �� � � � � � � � � � � �� � �� � � � � � � � �� � � �� �� � �� � � � � � � � � �� � ψk� � �
� � � � � �� � � � �� � �� �� � � � � � � � �� � � � �
B 3� � � � �� � � � B �# !� � �� � � ! + � � � " � � �� �+ � � � " � � �� �E �E � � � � �� F� � � � � � � � � � �� �
� � � � �� � � � X � � � � � � � � � {1, . . . , Q} � � � � �( � $ ! ��$ �� 4 � �� � � 5� $ � F� � � � � � (F)� � � ��
Ψ� � � � � �� � F � � � � � � � � � � � �� � � � � � Ψ = {ψk,l : 1 ≤ k 6= l ≤ Q}� �� � � � � �� � � �� ψk,l �� � �
�� � � � � 1� � � � �� �� � � � � � � � � � k� �� � � � � −1� � � � � � �� � � � � � � � � � l� � �� � � � � ∗
� �� � � � � � � � � � � � �
+� � �� $ � � ��$ �� "� � �( � $ ! ��$ &� � � � � �� � ! 8 �$ ! � �� � �� "� ( � �� � �� �� � � #�( � � ! � � ��$ � $ #�$ �� � �� � ! "�
� �� $ � � ��$ �� "� � �( � $ ! ��$ �� 4 � �� � � 5� $ ! 8 �$ ! � �� � $ � �� "� ( � �� � �� �� � � #�( � � ! � � ��$ � $ #�$ �� � � $ �
> � %� � "? @ @A E D BC E
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � � � �� � � � �� �� �� � � � � � � � � �� �� � � � �� � � γ
B 3� � � � �� � E D � B �# !� � �� � � � � � ) �� � � � !" �� � � ��� ! � " � � � , �� � � �" ! �E � � C � �� �� G� � � � � �
� � � � �� � � � � X � � � � � � � � � � � �� � � � γ ∈ R∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X
� � � �� � �� γ* � � " % �� �! � � �� G� � � � � �� � � � � � � vb = (bi) ∈ Rn � � � � �� � � � � � � � � � �
vy = (yi) � {−1, 1}n� � � � �� � � � � � �� gy � � � G� � � �� � �� � �
∀i ∈ {1, . . . , n} , yi (gy(xi) − bi) ≥ γ
� � � �( � $ ! ��$ '� �* ! � � � �� � �$ & 6( � � &� γ� � � Pγ � �( � $ ! ��$ � � �� � � � � � G� Pγ� � � � (G)� � � �
� � � � � � � �� � �� � �� � � � � � � � � � � � � X γ� �� � � � �� � � � � G� � � � � � � � � � � � � � �� � �� � � � � � �
� � � � � � �� � �� �
> � %� � "? @ @A E E BC E
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
γ� Ψ� �� � � � �� � � �
B 3� � � � �� � E E � γ ) Ψ ) � �# !� � �� � � � � � �� G� � � � � � � � � � �� � � � � � �� � � � X � � � � � �
� � � RQ � �� �� Ψ� � � � � � � � � � � � �� � � �� � ψ � {1, . . . , Q} � � � {−1, 1, ∗}� � � � � � �� � ∗
� �� � � � � � � � � �� �� � �� �� �� � � � � � � γ ∈ R∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} �
X � � � �� � �� γ* Ψ* � � " % �� �! � � Ψ* � � " % �� �! � � %� # � $ � ( � � &� γ � � � � ∆#G� � � � � �� � � � � � � �� � � ��
ψn =(
ψ(i))
1≤i≤n
� � � Ψn � � � � � � vb = (bi) � Rn � �� � � �� � � � � � � � � � � vy = (yi) �
{−1, 1}n� � � � �� � � � � � �� gy � � � G� � � �� � �� � �
∀i ∈ {1, . . . , n} ,
� � yi = 1, ∃k : ψ(i)(k) = 1 ∧ ∆#gy,k(xi) − bi ≥ γ
� � yi = −1, ∃l : ψ(i)(l) = −1 ∧ ∆#gy,l(xi) + bi ≥ γ
� � γ* Ψ* � �( � $ ! ��$ � � � Ψ* � �( � $ ! ��$ 6( � � &� γ� � ∆#G� � � � Ψ� � � � (∆#G, γ)� � � � � �� � � � � ��
� �� � �� � � � � � � � � � � � � X γ� Ψ� �� � � � �� � � �� ∆#G� � � � � � � � � � � � � � �� � �� � � � � � � � � � �
� � �� � �� � � � �� � �� $ � � ��$ ! � � � � � � � 6 #� " "� �� "� � �( � $ ! ��$ '� �* ! � � � �� � �$ & "� � ! �� � Q = 2 �
> � %� � "? @ @A E C BC E
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � � � � �� � � � �� � �� � �� � � � � � � γ
B 3� � � � �� � E C � B �# !� � �� � � ! + � � � " � � �� � # � " � ! γ � �� �� G� � � � � � � � � � �� � � � � �
�� � � � X � � � � � � � � � RQ �� � � � γ ∈ R
∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X � � � ��
� �� γ* 4 * � � " % �� �! � � � � �� � � � �� � � � � � � � � γ � � �� ∆#G� � � � � �� � � � � � �
I(sXn) = {(i1(xi), i2(xi)) : 1 ≤ i ≤ n}
� n� � � � � � � � � � �� � � �� � � � � � � � � {1, . . . , Q} � � � � � � vb = (bi) � Rn � �� � � �� � � � � � �
� � � � � � � �� vy = (yi) ∈ {−1, 1}n� � � � �� � � � � � �� gy � G� � � �� � �� � �
∀i ∈ {1, . . . , n} ,
� � yi = 1, ∆#gy,i1(xi)(xi) − bi ≥ γ
� � yi = −1, ∆#gy,i2(xi)(xi) + bi ≥ γ
� � � �( � $ ! ��$ �� 4 � �� � � 5� $ 6( � � &� γ � �� � � � � � ∆#G� � � � � � (∆#G, γ)� � � � � � � � � � � �� � �� �
�� � � � � � � � � � � � � X γ� � � �� � � � �� � � � � ∆#G� � � � � �� � � � � � � � �� � �� � � � � � � � � � �
� � �� � �� �
> � %� � "? @ @A E � BC E
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� �� � � � � � � � � � � � �� � � � � � � � �
� � � � � � � � � � � � � � �� � � � � � X � � � � RQ
$ !# # !E � � �� G� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � [−MG ,MG ]Q � � � � �
� � � � � � � � � � ε � � � ]0,MG ] � � � � � � � � � � � �� � n� � � �� � �� � � n ≥ � � � � � (∆G, ε/6)� �
� �� �� � � �� �� � � � � � � �
N (p)(ε,∆∗G, n) < 2
(
n Q2(Q− 1)
⌊
3MG
ε
⌋2)
l
d log2
“
enC2Q
“
2j
3MGε
k
−1”
/d”m
� d = � � � � � (∆G, ε/6) �
+� � � � � %� $ 8 � ! � � "� ! %� "� ) "� ! � "8 � � �� � �� � � ∆∗� ! � � � ( � "� # � �� � "8 � � �� � �� � � ∆ �
> � %� � "? @ @A E C BC E
� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�
� � �� � � � �� � � � � � � �� � � � � � � � � � � �
� � 3 � " � # ! � � � �� G �� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � [−MG ,MG ]Q �
� � � � � � � �� � � � �� � � � � � � � Q� � � � �� � � � � �� � � � � � � � �� �� δ ∈ ]0, 1[ �� � � �� � � � � � � �� � ��
� � � � � � � � � 1 − δ� � � � � � � � � �� � � � � � � � � � � � γ � � � ]0,MG ]� � � �� � � � � � � � � � ��
g � G � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �
R(g) ≤ Rγ,m(g)+√
√
√
√
√
√
2
m
ln
4
(
2m Q2(Q− 1)
⌊
12MG
γ
⌋2)
l
d log2
“
emQ(Q−1)“
2j
12MGγ
k
−1”
/d”m
+ ln
(
2MG
γδ
)
+
1
m
� d = � � � � � (∆G, γ/24) �
R(g) ≤ Rγ,m(g) + c ln (m)
√
d
m
� " � ��� � � � �� � E � * � � !" � !� � ! � � " ! � ! �� " !# !� � � � �� "# ! � �lim
m→+∞supP
P
(
supn≥m
supg∈G
(R(g) −Rγ,n(g)) > ε
)
= 0 limm→+∞
supP
P
(
supn≥m
supg∈G
|Rγ(g) −Rγ,n(g)| > ε
)
= 0
> � %� � "? @ @A E � BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
�� � � � � �� � � � � � � � � � � � �� � � � � � � �� � � � �
A �# � � � ! � ! �� � � � �� � � � ! � � � !
, � �� $ � κ� $ $ � :� � ! :( � �� � ��� � ! � ( �* � �� $ � � � ! � � �' ! � � X� � (Hκ, 〈·, ·〉Hκ) "� � � , #� � � � ! � �$ �� $ �
, � �� $ � H = (Hκ, 〈·, ·〉Hκ)Q� � H = ((Hκ, 〈·, ·〉Hκ
) + {1})Q
H � '� ( � " "� �� ! ' �$ # � ��$ ! h = (hk)1≤k≤Q �� X �� $ ! RQ �� " "� ! �� � �
∀k ∈ {1, . . . , Q} , hk(·) =
mk∑
i=1
βikκ(xik, ·) + bk
� %� # {xik : 1 ≤ i ≤ mk} ⊂ X (βik)1≤i≤mk∈ R
mk� � bk ∈ R � �$ ! � ��� � "� ! " �( � �� ! �� #� ! ' �$ # � ��$ !
"� � ! �� � "� ! � $ ! � ( ) "� ! {xik : 1 ≤ i ≤ mk} �� % �� $ $ � $ � �� $ ! � ! �� $ ! X� � ! � $ ! �� "� $ � �( � �$ � � � �� �� �
"� $ � :� �
A �# � � � ! � ! �� � � � �� � � " 3 � � � � � � � ! �
, � � ! * � $ ! � ( ) "� #�$ %�2 � �� H � � �� $ � �� � �� ! #�$ �� � �$ �� ! ! � � � $ ! � � ! * � ! �� #� � �$ � �
> � %� � "? @ @A E � BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
� � � � � � � � � � � � � �� � � � � � � � � �
� ( � � � ! � � � � � �
� � � �� � �$ � :� � �� = � � #� � κ � " �2 �! �� � $ � ' �$ # � ��$ Φ �� " "� �� � �
∀(x, x′) ∈ X 2, κ(x, x′) = 〈Φ(x),Φ(x′)〉
�� 〈·, ·〉� ! � "� � � � � � � � ! #� "� �� � �� "8 � ! �� #� `2 � , � � � Φ (X ) = {Φ (x) : x ∈ X}
� � � � � � � �� � � � � � �� � "8 � $ �� ! � ! �� #� ! �� � � " )� � � (EΦ(X ), 〈·, ·〉)� $ &� $ �� �! �� � "� ! Φ (X )
=⇒ H ��� � � � �� � #�$ ! � � �� �� #�( ( � � $ � '� ( � " "� �� ' �$ # � ��$ ! � �$ � ! ( � " � � %� � � �� ! ! � � Φ (X )
h(·) = (〈wk, ·〉 + bk)1≤k≤Q
w = (wk)1≤k≤Q ∈ EQΦ(X ) b = (bk)1≤k≤Q ∈ R
Q
+ � "# ! � � " H ! � EQΦ(X )
∥
∥h∥
∥
H=
√
√
√
√
Q∑
k=1
∥
∥hk
∥
∥
2
Hκ=
√
√
√
√
Q∑
k=1
〈wk, wk〉 =
√
√
√
√
Q∑
k=1
‖wk‖2 = ‖w‖
‖w‖∞ = max1≤k≤Q
‖wk‖
> � %� � "? @ @A E BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
Q ≥ 3 �� � � � � � � � � � � � � �� � �� � � � �� � � ���� � � � � � � � � �� � � � �
((xi, yi))1≤i≤m ∈ (X × {1, . . . , Q})m � � � � � � � � � �� � �� � � �
` �� �� � � ' �$ # � ��$ �� �� � �� #�$ %�2 � � #�$ ! �� � � �� � � �� � � �� "� ' �$ # � ��$ �� ��� � �� # � � �$ �1 � � �
. ) ,- . � �� � � �� � � � � � " � � �� # ! � ! � " � � " �# # � � �� � �� � !� ! � � � " � � � ! �
� " � � �� # !E
minh∈H
{
m∑
i=1
` �� � � (yi, h(xi)) + λ‖h‖2H
}
� �� �
∑Qk=1 hk = 0
� � 3 � " � # ! � ! " ! � " 3 � !� � � � �� �
� � � �� � 1 ( � � �� ) " � � �� � "8 � � � � � $ � �! ! � &� � "� � �! � "� � ��$ � � � � � ) "1 ( � � � � � % �� $ � 6 �� � � %� � "� ! %� "� � � !
�� ! #� � � # �� $ �! βik �� $ ! �
∀k ∈ {1, . . . , Q} , hk(·) =m∑
i=1
βikκ(xi, ·) + bk
� "� ! %� "� � � ! �� ! ; ) �� �! ; bk! 8 � $ � � � � �! � $ � �� � � � � " � #� � ��$ �� ! #�$ � � � ��$ ! �� � � � $ * 0 � # � � �
> � %� � "? @ @A E E BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
�� �� � � � � � � � � � � � � � �� � � �
( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !
� " � � �� # !C � . ) ,- .E � - � � � �� � � � ��� �E � �E ��� ! � �� � � � � � � �� � �E � �E ��� � � �
minh∈H
1
2
Q∑
k=1
‖wk‖2 + C
m∑
i=1
∑
k 6=yi
ξik
� �� �
〈wyi− wk,Φ(xi)〉 + byi
− bk ≥ 1 − ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
> � %� � "? @ @A E � BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
�� �� � � � � � � � � � � �� � � � � � �� � � �� � �
. � " � ! � � 3 � # 3 �" � ! �d �� � � � = min
1≤k<l≤Q
{
min
[
mini:yi=k
(hk(xi) − hl(xi)) , minj:yj=l
(hl(xj) − hk(xj))
]}
∀(k, l), 1 ≤ k < l ≤ Q,
d �� � � � ,kl =1
d �� � � �
min
[
mini:yi=k
(hk(xi) − hl(xi) − d �� �� � ) , minj:yj=l
(hl(xj) − hk(xj) − d �� � � � )
]
∀(k, l), 1 ≤ k < l ≤ Q, γkl = d �� �� �
1 + d �� � � � ,kl
‖wk − wl‖
$ � !� !� �" ! � ! � 3 � � � � � � � ! " ! � � ! �# � " � ! � � 3 � # 3 �" � ! �
∑
k<l
‖wk − wl‖2 = Q
Q∑
k=1
‖wk‖2 −∥
∥
∥
∥
∥
Q∑
k=1
wk
∥
∥
∥
∥
∥
2
∧Q∑
k=1
wk = 0 =⇒
Q∑
k=1
‖wk‖2 =d2�� �� �
Q
∑
k<l
(
1 + d �� � � � ,kl
γkl
)2
> � %� � "? @ @A C D BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
�� � � � � � � � � � � � � �� � � �
( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !
� " � � �� # ! � � . ) ,- .C � * " �# # !" � , �� � !" �C D DE �
minh∈H
{
1
2
Q∑
k=1
‖wk‖2 + Cm∑
i=1
ξi
}
� �� � 〈wyi− wk,Φ(xi)〉 + δyi,k ≥ 1 − ξi, (1 ≤ i ≤ m), (1 ≤ k ≤ Q)
( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � � � !
, � �� $ � αi. = (αik)1≤k≤Q δyi,. = (δyi,k)1≤k≤Q τi. = (τik)1≤k≤Q = Cδyi,. − αi.� �
τ = (τik)1≤i≤m,1≤k≤Q
� " � � �� # ! C � . ) ,- .C �
minτ
1
2
m∑
i=1
m∑
j=1
τTi. τj.κ(xi, xj) −
m∑
i=1
τTi. δyi,.
� �� �
τik ≤ Cδyi,k, (1 ≤ i ≤ m), (1 ≤ k ≤ Q)
1TQτi. = 0, (1 ≤ i ≤ m)
> � %� � "? @ @A C E BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
�� �� � � � � � � � �� � � � � � � � �
( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !
� " � � �� # ! � � . ) ,- . � � $ ! ! �� �� � �C D D C �
minh∈H
1
2
Q∑
k=1
‖wk‖2 + C
m∑
i=1
∑
k 6=yi
ξik
� �� �
〈wk,Φ(xi)〉 + bk ≤ − 1Q−1 + ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)∑Q
k=1 wk = 0,∑Q
k=1 bk = 0
& 3 � � � � � � ! �� � � � � � �� � ! ��� � �� � �C D D C � � !� � " � � � � " � � ! � � �C D D �
� � �� = * ,< = � ! � "� ! � � "� � �$ � "8 � � � � � $ � �! ! � &� ! � � �� � :� ! * #�$ ! �! �� $ � �
> � %� � "? @ @A C C BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
� �� � �� � � � �� � � �
� " � � �� # ! � � . ) ,- . � ! � ! � �� � ! � � � � � �� � � � � � � �� � � � � � "# ! ‖ · ‖∞ �
minh∈H
1
2t2 + C
m∑
i=1
∑
k 6=yi
ξik
� �� �
〈wyi− wk,Φ(xi)〉 + byi
− bk ≥ 1 − ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
‖wk‖ ≤ t, (1 ≤ k ≤ Q)
Mξ =((
δk,l − 1Q
)
δi,j
)
1≤i,j≤m,1≤k,l≤Q
� " � � �� # ! � . ) ,- . � ! $ ! ! � $ �� ! � � � � � � �� �� � � � � " � � � ! � �
minh∈H
{
1
2
Q∑
k=1
‖wk‖2 + CξTMξξ
}
� �� �
〈wk,Φ(xi)〉 + bk ≤ − 1Q−1 + ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)
∑Qk=1 wk = 0,
∑Qk=1 bk = 0
> � %� � "? @ @A C � BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
� �� � � � �� � � � ��� � � � �� � � � � �� � � � � �
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
x_2
x_1
� � � � ? � � #� � � & � � �� ! " �$ �� �� � ( � $ � ! � �� � � ) "� ! �� $ ! R2
> � %� � "? @ @A C C BC E
� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !
� �� � � � �� � � � ��� � � � �� � � � � �� � � � � �
� � � � � � � : ��� � � "� $ ! ! � �� � � �� � � ! � �( � � &� ! � � � #� ! �� "� = * ,< = " �$ �� �� �
> � %� � "? @ @A C � BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� � � � � �� � � � �� � �� � �� � � � � � � � � � �� � � �
� � 3 � " � # ! C � � �� H �� � � � � � � � � � � �� � � � � � �� � � � � � � � � � � � � � � Q� � � � �� � � � � � � �
���� � �� �� � � � Φ (X ) � � � � �� � � � � �� �� � � � � � �� ΛΦ(X )� �� � � � � ��� � � � � � EΦ(X )� � �
� � � � w � � � � ‖w‖∞ ≤ Λw � � b = 0 �� �� � � � �� � � � � � � ε ∈ R∗+�
� � � � �(
∆H, ε)
≤ C2Q
(
ΛwΛΦ(X )
ε
)2
+� � � � � %�
� $ 8 � ! � � "� ! %� "� ) "� ! � "8 � � �� � �� � � ∆� ! � � � ( � "� # � �� � "8 � � �� � �� � � ∆∗
� ! 8 � � � � �� � �� � # �� ( � $ � ! � � "� � � �$ # � ��� �� � � #�( � � ! � � ��$ � $ #�$ �� � � $ � � "� � � �$ # � ��� �� ! � �� � �� !
� � � �� " "� "8 � � � " �! � � ��$ �� "� $ � �( � ‖ · ‖∞� �$ �$ #� " "� �� "� $ � �( � ‖ · ‖ �� � � " �! �� �� � "� � �$ � " �! � �� � � �
Q = 2 : Pε* � �( (Hκ) ≤(
ΛwΛΦ(X )
ε
)2
> � %� � "? @ @A C � BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� �� � � �� �� � � �� � � �� � � � � � � � � � � � �� � � � �
B 3� � � � �� � E � � . � � !� � ! � ! & � � !# � �� !" � � � � � n ∈ N∗� � � � � A� � � � � �� � � �
� � � � � a = (ai)1≤i≤n � � � � � � � � � Rn � (σi)1≤i≤n� � � �� � � � � � � � � � � � � ( � :� $ $ � ��
� �� ( � # � � � � ! ! � # � �� 6 A� Rn(A)� � � �� � � � � � �
Rn(A) = E supa∈A
1
n
∣
∣
∣
∣
∣
n∑
i=1
σiai
∣
∣
∣
∣
∣
� � 3 � " � # ! � � '� 3 � � � � � 3 � ! � � �� 3 " !� � ! � �� " � 3 ! � � . � B � � "# � � �E �E � � � � � � n ∈ N∗� � � ��
(Ti)1≤i≤n� � � �� � n � � � � � � � � �� � � � �� � � �� � � � � � � � � � � � � � � � � � � � T � �� �� g
� � � � �� � T n � � � R � � � � � � � � �� � � � � �� � � � � � � � � �� � �� � � (ci)1≤i≤n � � � � � � �
∀i ∈ {1, . . . , n} , sup(ti)1≤i≤n∈T n,t′i∈T
|g(t1, . . . , tn) − g(t1, . . . , ti−1, t′i, ti+1, . . . , tn)| ≤ ci.
� �� � � � �� � � � � � � τ ∈ R∗+� �� � � � � � � � �� � � � �� g (T1, . . . , Tn)� � � �� � �� �
P {g (T1, . . . , Tn) − Eg (T1, . . . , Tn) > τ} ≤ e−2τ2
c
P {Eg (T1, . . . , Tn) − g (T1, . . . , Tn) > τ} ≤ e−2τ2
c
� c =∑n
i=1 c2i �
> � %� � "? @ @A C BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� �� � � �� �� � � �� � � �� � � � � � � � � � � � �� � � � �
& � � ! � # � " � ! �� � !� �� 3 � � �� � �3 � � � . ) ,- . � ! * " �# # !" ! � , �� � ! "
R(h) = E[
(1 − ∆hY (X))+]
� � 3 � " � # ! � � � �� H �� � � � � � � � � � � �� � � � � � �� � � � � � � � � � � � � � � Q� � � � �� � � � � � � �
���� � �� �� � � � Φ (X ) � � � � �� � � � � �� �� � � � � � �� ΛΦ(X )� �� � � � � ��� � � � � � EΦ(X )� � �
� � � � w � � � � ‖w‖∞ ≤ Λw � � b = 0 � �� �� KH = ΛwΛΦ(X ) + 1 �� � � �� � � � � � � �� � �� � � � �
� � � � � 1 − δ� � � �� � � � � � � � � � �� h � H � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �
R(
h)
≤ Rm
(
h)
+4√m
+4Q(Q− 1)Λw
m
√
√
√
√
m∑
i=1
κ (Xi, Xi) +KH
√
ln(
1δ
)
2m
R(
h)
≤ Rm
(
h)
+O
(
√
1
m
)
> � %� � "? @ @A C E BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� � � � � � � � �� � � � � �
� � 3 � " � # ! �- � � � �� �E � �E � � � � �� � � � � � � � � �� � �� � � � � �� � �� � � �� �� Lm� � � � ��
� � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � � � � � � γ = 1‖w‖� � � �� � �� � � � �� � � � � �
��� � � � � � � � � �� � �� � � � � � � � � � � �� � � � � �� � � � � Lm �
Lm ≤ D2m
γ2
� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �
� � � �� � � �
> � %� � "? @ @A C � BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� � � � � � � � �� � � � � � � �� � � �� �� � � � � � � � � � � � � � �� � � �
d � � = d � � = 1
� � 3 � " � # ! E � � � �� � � � � � � � � � � � � � � � � � �� � � �� � � � � � � � � � � � � � Q
� � � � �� � � � � � � � � �� � � � � �� Lm � � � �� � � � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � �
� �
Lm ≤ K�Q
D2m
∑
k<l
(
1 + dkl
γkl
)2
� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �
� � � �� � � �
* � � � � �� � ! K ��� +� %� "� � � �� K� � ! 8 � ) � �� $ � �� � � �! � "� � ��$ �� � � � &� � ( ( � ! �� � �� � � � �� � ! � $ $ �( ) � � � &� "� �
$ � ( ) � � �� %� # �� � � ! ! � � � � � �
� � � � Q = 2 K� � = 2 � � "� ) � �$ � ! � � � � � � � 6 "� ) � �$ � ;� � : �$ *( � � &� ; ) �* # "� ! ! �
> � %� � "? @ @A � D BC E
� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !
� � � � � � � � �� � � � � � � �� � � �� �� � � � � � � � �� � � � � � � � �
d � � � = QQ−1
� � 3 � " � # ! � � � � �� � � � � � � � � � � � � � � � �� � � � � � Q� � � � �� � � � � � � � � �� � � �� �� Lm
� � � �� � � � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � � � �
Lm ≤ D2m
∑
k<l
(
1 + d � � � ,kl
γkl
)2
� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �
� � � �� � � �
� � �� ) � �$ � ! � � � � � � � � $ #� � � 6 "� ) � �$ � ) �* # "� ! ! � � � � � Q = 2 �
> � %� � "? @ @A �E BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
�� � �� � � � �� � � � � � �� � � � � � � � � � � � � � � � � � � � �
* � � � !� � ! � �� �� � � ! �2 � "� � �� � ��$ ' �$ # � ��$ $ � " "� �� ! �$ ' � �( � � ��$ ! � � � %� $ � $ � �� ! &� � $ � !
� � � &� � ( ( � ! �� ! � �� � $ �� &� �� ! & �$ �( � ! � �� ! ! � �� � "� #�$ $ � �! ! � $ #� �� "� ! �� � # �� � � � � �� ! � � � � � �$ � ! �
� � � � � � % �� ( � ! ! � %� �� ! � ��� � $ #� ! � � � � � � �� � ! � #� � �! ! � $ #� �2 � �$ � $ � �� " "� �� ! )� ! � ! �
�� �� � � �� �� � �� � � � � � � ��� � � �� � � �� � � � � �� �� � � � � � �� �� � ��
? � � � �� �( �$ � � ��$ �2 � �� �( � $ �� "� �� "� ! �� � # �� � � � � � � # � � �� 1 ! "� � � �� � � � "� � ! �� 8 � " "� � ! � � �� " �! � ) "�
=⇒$ � #� ! ! � � � �� �� ! ! � � �98 � $ � � � � � � # � � ) �� # � �( � ��� � 6 � $ � � � � � � # � � � � � � � # � � %�
� � ) "1 ( � #� $ �� � " � $ ) �� "� & �� ��� �( � � �� $ � �98 � ) � � �� �
"8 � ! ! � $ � �� " �� ! &� � $ �� ! �� � ! � ��$ ! � � %� � �� ! � $ �� � � �� ( � $ � �� � �$ $ �� ! ! � ��� � $ � �� " "� !
> � %� � "? @ @A �C BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � � � � � � �� � �� � � ��� � �� �� �� �� � � � � � � �� � � � � � � � � � � � � �
• , 3 !� � ! � � �" � � " ! � " �# � �" ! � 1.6 · 106 �3 !� � ! � �� � � ! � �
�� �� �� � �� � �� � � � � � � �� � � � � � �� � � � �4 4 4
• , �" � � " ! � ! �� � � � �" !
�� �� � � � �� � � � � �� � �� � � �� ��� � � � � � ��� � �� �� � � α � � � � � � � � � � �� � � β � � �� �� �� �
• , �" � � " ! � !" � � � �" ! � 2.7 · 104 � �" � � " ! � � B �� � � ! � �
> � %� � "? @ @A � � BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � � � � � � � � �� �� � � �� � �� � � � �� � � � � � �� � � �� � � � � � �
� � � �� � � � � �� � � ! � � �3 !� � ! � ! � !x = (xi)−n≤i≤n � %� # �� � � #� �� $ � � $ � � " : ��� � � � �� � #�$ �� $ � �98 � $ � '� $ � �� � �98 � $ � " : ! � �� �� � " "� 2n+ 1 �
κθ,G (x,x′) = exp
(
−n∑
i=−n
θ2i ‖xi − x′i‖2
)
� � � �� � � � � �� � � ! � � � � � � !# !� � �# � � � � � ! �
x = (xi)−n≤i≤n �� " �� � xi =∑22
j=1 θijaj � #� ( ) �$ � �! �$ #�$ %�2 � �
〈xi, x′i〉 = 〈
22∑
j=1
θijaj ,22∑
k=1
θ′ikak〉 =22∑
j=1
22∑
k=1
θijθ′ik〈aj , ak〉
> � %� � "? @ @A � C BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � � � � � �� � � � � �� � � � �� � � � � �� � � �� �� �� � � � �� � � � � � G �
� � � � �
� S �( � �� � #� �� ! � ) ! � � �� � ��$ � � � � �� 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � � $ ! �� �� �� � �
E �E � �> � %� � "? @ @A � � BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � � � �� � � �� � � � � S � � � � �� � � � � � � � � � �
� A = (aij) ∈ M22,22(R) �� � � � �! � $ �� � ��$ %� # �� � �� " "� �� ! � # � �� ! � ( �$ �! �� $ " � &$ � �� G = AAT �( � �� � #� �� ! � � � � � � �! ! #� "� �� � ! � � � � � �2 �( � � ��$ �� S �� � � $ � ( � �� � #� ! :( � �� � �� �
! � ( �* � �� $ �� � � ! � � � %�
� �� & �$ � " �! � � ��$ �� S �
S = PDP−1 = PDPT
� � ! � � � �� � & �$ � "� � � �! �� � S� ! � ! :( � �� � �� � �AAT = PD+P
T
�� D+� ! � � � � � � �� �� D� $ � � ( � "� �� $ � �� � 0 "� ! %� "� � � ! � � � � � � ! $ � &� � � %� !
�$ � $ � � � � � �
A = P√
D+
> � %� � "? @ @A � � BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � � � � � � � � � � � �� � � � � � � � �� � � θ
( � � � � !# !� � � ! � � � � � � * " � � � � �� �� � �� �� � �C D DC �
A(κ, κ′) =〈κ, κ′〉‖κ‖‖κ′‖ =
∫
X 2 κ(x, x′)κ′(x, x′)dPX (x)dPX (x′)
√
∫
X 2 κ(x, x′)2dPX (x)dPX (x′)√
∫
X 2 κ′(x, x′)2dPX (x)dPX (x′)
( � � � � !# !� � � � � � ) � � � � ! !# � �" � !
ADm(Kθ,G,Kt) =
〈Kθ,G,Kt〉F‖Kθ,G‖F ‖Kt‖F
θ∗ = argmaxθ∈Θ
ADm(Kθ,G,Kt)
> � %� � "? @ @A � BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � �1� � � � � � � � �� � � �� �� � � � � � �� � � � � � ���� � � � � � � �� � � ��� � �� �
�� �� � � Q = 3 � � � � � �! � $ �� $ �! � � � �( � �2 �� ! #� � � & � � �� !
! � y = y′, κt(x, x′) = 1
! � y 6= y′, κt(x, x′) = − 1
Q−1
> � %� � "? @ @A �E BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� � �� � � �� � � � � � �� � �� θ∗
0
0.2
0.4
0.6
0.8
1
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
thet
a
position dans la fenetre
"weights.txt" using 1:2
� � � � A � < � # �� � � θ� ) �� $ � �� � � " � &$ � ( � $ �$ � :� � * # � ) "�
� " & � � � �� ( � �98 � � � � � $ � �! ! � &� � �� ! #� $ �� � $ &� � � �� $ � ! �� # � � ! � � �� �
> � %� � "? @ @A � � BC E
� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !
� �� � �� � � � � � � � � � �� � �
< � " � �� � ��$ #� � �! �� 6 � �� ! ! � � � $ � )� ! � �� E D � � � " � � 3 �� ! � #�$ ! � � �� �� ! �� C �E � � " 3 � � � �
� �� �2 �98 � �� $ � � � � � � @ � �! � ��� � $ #� � " � &$ � ( � $ � * � � � � " � " � &$ � ( � $ � * ! � � � ��
= = * ,< = = = * ,< = = = * ,< =
Q3 � � � � �? � � �? � @ �? � � � A � > � > � �
Cα @ � � � @ � � � @ � � � @ � � � @ � � � @ � � >
Cβ @ � � � @ � � A @ � � � @ � � � @ � �? @ � � A
Cc @ � �A @ � � � @ � � � @ � � � @ � � � @ � � �
Sov � � � > � � � � � � � � � � � � � � � @ � � �A
Sovα � � �A � � � > � � � � � � � � � � � � � � � @
Sovβ � � � � � � �A � � � � �? � � �A � � � � � �
Sovc � � � � � � � � � � � � � � �A � � �? � � � �
> � %� � "? @ @A C D BC E
� ��� � ��� �� �$ # "� ! ��$ ! � � ��� � ! ��� # � � %� !
� � � � � � �� � � � � � � � � � � � �� � � �
& � � ! � � � " �� � � � ! � �3 � ! � � �� � � !# � �� � ! ��� " � ! � . ) ,- .
� + � ! γ* Ψ* � �( � $ ! ��$ ! 5� � � $ � � � � � "� ! = * ,< = �� � "� ! = � � "� ( �( � � � "� �� � "� � �( � $ ! ��$
'� �* ! � � � �� � �$ & � � � � "� ! ,< = ) �* # "� ! ! � ! �
� � 8 �( � � � �� $ �! � � � &� 1 ! � � ! �� $ � 6 � � � # �� � � �� $ ! "� ' � �( � "� � ��$ �� "� ) � �$ � < � � "� ( � 5� � � � ��$ �� !
( � ! � � � ! �� #� �� # � � � �
� +� ! � "� # � ��$ �� ( � �1 "� ' � � �$ � � � $ � �� � � � �� �� � # � � � � � � "8 � %� "� � � ��$ �� ! � �! �� � ! &� � � $ � �! � � �� !
) � �$ � ! ! � � "� � �! �� � � ( � �� � �� � ��� � $ � � ! � �� ) " �! ! �$ ! �
� " 3 � � � � �� � � ! � � � �" � � " ! � ! �� � � � �" ! � ! � � " � � 3 �� ! �
� + � ! = * ,< = � � � %� $ � � � � % � �� � �� � � � � " �! �� ! � � # � � � �� ! = #�( ( � � " �( � $ �! �� )� ! � �� !
( � �� � �� �� � � � � � # � ��$ �� "� ! �� # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � ! �
� + � � � %� "� � �� ( � $ � �� ( � �1 "� ! � : ) � � �� ! �$ � � &� � $ � ! : ! �1 ( � ! � �! #� �( �$ � $ �! � � & �$ �� � � �' ! #�$ ! � � �� �
"8 � $ � �� ! � � �$ # � �� "� ! � � � ��$ ! � � � � � ) �� $ �� �� ! � � � &� 1 ! � $ ) �� "� & �� ! �� � # �� � � "� � � � � � # � � %� �
> � %� � "? @ @A CE BC E
� ��� � ��� �� � � � � $ ! � % � �� � "� !
�� � � �� � �� �� � �� � �
> � %� � "? @ @A CC B �
� ��� � ��� �� � � � � $ ! � % � �� � "� !
� �� � � � � � � � � � � � �� � �
$ !# # !C �- � � � �� � * � !" � � !� � � � �E � E � , � !" �E � C � , � ! � � � �E � C � �� � � F�
� � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � �� � � ΠF� � ' �$ # � ��$ �� #� � �! ! � $ #� � � � �
n ∈ N∗ � � � � � � ΠF (n) = supsXn⊂X N (F , sXn) � d� � � �( � $ ! ��$ < � ΠF (d) = 2d �
ΠF (d+ 1) < 2d+1 � �� �� � � �� � � n ≥ d�
ΠF (n) ≤d∑
i=0
Cin <
(en
d
)d
� e � � �� � � � � � �� � �� ��� � � � � � � � � �
> � %� � "? @ @A C � B �
� ��� � ��� �� � � � � $ ! � % � �� � "� !
� �� � � � � � � � � � � � �� � � � � � � � �
� � � � � � � � � � � � � � �� � � � � � X � � � � {1, . . . , Q}
$ !# # ! � � � � � � � !" � $ � � � �E � � � � �� � � F� � � � � � � � � � �� � � � � � �� � � � X �
� � � � � � � � {1, . . . , Q}� ΠF� � � � � �� � � � � �� � � � � d� � � � � � �� � � � � �� � � � � � �� � � �� � �
n ≥ d�
ΠF (n) ≤d∑
i=0
Cin
(
C2Q+1
)i<
(
(Q+ 1)2en
2d
)d
> � %� � "? @ @A C C B �
� ��� � ��� �� � � � � $ ! � % � �� � "� !
� �� � �� � �� � ���� � � � � �� � � � � ��
� � � � � �� �� �� �� � �� �� � X� � � � R
� �� � �� ���� �� ! "#$ %'& ( () *,+ -./ G 012 3546 .7 72 82 3 -19 / . -1: : 0; 01 8 -6 4 . 12 X <= 4 72 0;: 8 4 1 :
[0, 1]> ? - 0; / - 0/ 2 = 4 72 0; 8A@ ε 8 4 1: ]0, 1]2 / / - 0/ 2 = 4 72 0; 2 1 / .B ;2 82 n: 4 / .: 354 .: 4 1 / n ≥ Pε/4
C 8 .6 (G) D -1
8 .: E -: 2 82 7 4F -; 1 2 : 0 .= 4 1 / 2 G
N (ε,G, n) < 2
(
4n
ε2
)d log2(2en/(dε))
-H d = Pε/4
C 8.6 (G)>
IJK LMNO P PQ � R S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � �� �� �� � � � � � � � � � � � �� �� �� � � � � � � � � � � �� � � � � � � �� � � � ��
� �� � �� � ��� ���� �� �� � �
���J � �N M ��� �� M L� ! �" � �! � � � �J # L" � M � M! MJ N � $ J L N J � M" � � �M � % LJ �'& M �� �� � F� �! �M" �N � " � �! N J
� M" � � �M � ( �� � N �2 : E 49 2 82 : ); 4 E�*2 : � � F $ GF
+,- � � .� .� �/ �� 0 �� 1 �� / �� �2 3 � � � �
4 � , � � 5� � & 6 �� � .� �� 7 % & ( () *+ -./ ΠGF7 4 3 -19 / . -1 82 9 ; -.: : 4 19 2 82 7 @2 : E 49 2 82 : ); 4 E*2 : 82
F> 87 -;: G
PDm
(
supf∈F
(
R(f) −Rm(f)√
R(f)
)
> ε
)
≤ 4ΠGF (2m) exp
(
−mε2
4
)
IJK LMNO P PQ � 9 S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � � � � � �� �� � � �� � �� � �� � � � � � �� �� �� � �
� M � �! X� � � � �J � �� � N � �� � �! (H, 〈·, ·〉H)� � � � �J �� � � M N �� L! � � # � ! M � � �� L X � H ⊂ RX ��
� 7�� � � � � �� � 2 - � � . ��� � � �- �� � % & ( R 6 *
� M! κ� �� # � ! M �� � X ×X� J � � R
∀x ∈ X $ � M! κxN J # � ! M �� � X� J � � R� �� � M � �J L κx : t 7→ κ(x, t)
κ� �! J � ��� N � � � 1 - � 4 0 ;2 E ; - 8 0 .: 4 1 / � � L H �M �! �� � N � " � �! �M�
� � ∀x ∈ X , κx ∈ H
O � ∀x ∈ X , ∀h ∈ H, 〈h, κx〉H = h(x) � E ; - E ; .� / � 82 ;2 E ; - 8 0 .: 4 19 2 �
�- � � / � � �� 2 � � �� .� � � 7�� � � � � �� � 2 - � � .
� M � � � �J � L� � L � � M �J �! �� M �! � $ H� �! J � ��� N � � �2 : E 49 2 82 � .7F 2 ; / < 1 - � 4 0 ;2 E ; - 8 0 .: 4 1 / ���� � � ��
� � � � �J � � � �" �! LM �� � � �� " M �� �� � M � � �M! M # � � �! � � � � �J � � L� � L � � M �J �! � �
IJK LMNO P PQ � ) S R)
� ��� M ��� �� � � L � � �JK M L �N � �
Q = 2� � � � � � � � � � � �� � � � � � � � � � � � �� � � � � � � � � �� � �
((xi, yi))1≤i≤m ∈ (X × {−1, 1})m� 2 1: 2 6 F 72 8A@ 4 E E ;2 1 / .: : 4 )2
h = (h1, h2) = (h1,−h1) $ h(x) = h1(x) = ∆#h1(x) = 12 (〈w1 − w2,Φ (x)〉 + b1 − b2)
` ��� (y, h(x)) =(
1 − yh(x))
+
� 3 -19 / . -1 82 E2 ; / 2 9 * 4 ; 1 .B ;2 �
�� - �� � .2 �� � �� � � � � �� 5� �� � � � � 1 � � � � � .2 �� / � � 0 �� � � � � � � � .2 � � *
� � � �� 5 � � �
minh∈H
{
m∑
i=1
` ���(
yi, h(xi))
+ λ∥
∥
∥
¯h∥
∥
∥
2
Hκ
}
4 � , � � 5� �� � � � � � ,- �� .� .2 � �
� �! & � L �" � �! J �N M! �� � N �J � � L� �! M � �J %� �N J L � � N �! M �� � � L �N �" � Q � L�K M � �! �! L �K � L N � �K J N � � L �
� � � �� M � �! � βi� J � ��
h(·) =
m∑
i=1
βiκ(xi, ·) + b
�N J K J N � � L� � � � MJ M � � b �� � �� �� � M! �J LJ � �N M J! M �� � � �� M! M � �� � � � & �� � � �� L �
IJK LMNO P PQ � � S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � �� � � � � � �� �� � � � � �� � � �
�� 1 � �2 .� � � � �� � � � �� .2 - - � 1 �� 3 � � � � � � .2 �� � � �� �
αik� " � N! M �N M J! � � L� � �J % LJ � %� J � � M � � N J �! LJ M �! � 〈wyi− wk,Φ(xi)〉 + byi
− bk ≥ 1 − ξik
α = (αik)1≤i≤m,1≤k≤Q $ (αiyi)1≤i≤m = 0
� � � �� 5 � � ( � � �� & *
minα
{
1
2αTH � � α− 1T
Qmα
}
: > 9 >
0 ≤ αik ≤ C (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)∑
i:yi=k
∑Ql=1 αil −
∑mi=1 αik = 0 (1 ≤ k ≤ Q− 1)
H � � =((
δyi,yj− δyi,l − δyj ,k + δk,l
)
κ(xi, xj))
1≤i,j≤m,1≤k,l≤Q
w∗k =
∑
i:yi=k
Q∑
l=1
α∗ilΦ(xi) −
m∑
i=1
α∗ikΦ(xi)
IJK LMNO P PQ � ( S R)
� ��� M ��� �� � � L � � �JK M L �N � �
C_1 / C_2C_2 / C_3
x_1x_2
x_3
�� �� I � � J! � % LM � � � � N M � �J M L�" � �! � � �J LJ �N � �� J � � R3
IJK LMNO P PQ R 6 S R)
� ��� M ��� �� � � L � � �JK M L �N � �
C_1 / C_2C_1 / C_3C_2 / C_3
x_1x_2
x_3
� � � � � P � � � ��� L �N J � � � � �J LJ! � � L � �! K � ! � � L � �� � � L! � � N J � � � ( � N M � �J M L�
IJK LMNO P PQ R& S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � � � � � � � � � � � � � �� � � � � � � � � � � � � � � � � �� � � � �� � �� � � � � � �� � � � � � � � � � � �
∆ 6= ∆∗ � � L Q ≥ 3
� �� � �� � ��� ���� �� �� � � 1, � , � �� 2 - ,
� � � LJ! � � L ∆� # � L � M! �! L � �� � M � # L" J! M � � � L �! J �N M L N � N M � � � �! L� � � �J LJ! M � �! J �J M! � �� � � NK � LM �� L � � � �M � %N �! �
�2 � ��- 2 �� � � 1, � , � �� 2 - , �
� � � LJ! � � L ∆∗� �� # � L � M! �J �J � ��� � � M � # L" J! M � � � L J N � N � L � �� � L �� �� L � ��� M" � � �M � ( �
% � � � LJ N M � ��
=⇒J �! � � � L �J N M �� L N J ! LJ � �M! M � � �! L� ∆∗�! ∆J � � MK � J � � � N � " " �� � �J � � L� �& � N J & % � � � LJ N M � �
IJK LMNO P PQ R� S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� �� � � � � � � � � � � � � �� � � � � � �� � � � � � � � ��� �� � � � �� �
� , � � 2 .2 �� & � � � � � � �- � � �� .� � �2 � � �� � ��- �� �� � *+ -.2 1 / (E, ρ) 01 2 : E 49 2 E: 2 0 8 -C6 � / ; . � 02
� - 0 (E, ‖ · ‖E) 01 2 : E 49 2 82 � 4 1 49 * �2 / E′ 01 : - 0: C2 1: 2 6 F 72 F -; 1 � 82 E> 87 -;: D E - 0; n ∈ N∗ D72
nC .B 6 2 1 -6 F ;2 8 @2 1 / ; - E .2 82 E′ D εn (E′) D2 : / G
εn (E′) = inf {ε > 0 : N (ε, E′, ρ) ≤ n}
� , � � 2 .2 �� & R � � � � � �- � � �� .� � �2 � � �� � � �, � � . �� � � 2 � , �2 � � � � � � , *+ -.2 1 / (E, ‖ · ‖E)2 /
(F, ‖ · ‖F ) 82 0� 2 : E 49 2 : 82 � 4 1 49 * > + -./ L(E,F )7 @2 : E 49 2 82 � 4 1 49 * 82 / - 0: 72 : - E� ; 4 / 2 0;:
�7 . 1 � 4 . ;2 : F -; 1 � : � 82 (E, ‖ · ‖E) 8 4 1: (F, ‖ · ‖F )6 01 . 82 7 4 1 -; 6 2 G
∀S ∈ L(E,F ), ‖S‖ = supe∈E:‖e‖E=1 ‖S(e)‖F>
εn(S) = εn(S(UE))
IJK LMNO P PQ R S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� �� � � � � � � � � � � � � �� � � � � � �� � � � � � � � ��� �� � � � �� �
� , � � 2 .2 �� & 9 ��� �, � � . �� � � �, 0�� � � .2 �� * ? - 0; n ∈ N∗ D: -./ xn ∈ Xn> � @ � � LJ! � � L� � �K J N � J! M �
Sxn: 0; H2 : / 8� � 1 . E 4 ; G
Sxn : H −→ `Qn∞
h = (wk)1≤k≤Q 7→ Sxn
(
h)
= (〈wk,Φ(xi)〉)1≤i≤n, 1≤k≤Q
U ={
h ∈ H : ‖w‖∞ ≤ 1}
� � N M � � � �! L� N (ε,U , n)�! N � � � " � L� �� � � �! L � M �� � Sxn� �! # � L � M �J L N J � L � �M! M � �� MK J �! � �
� � � � �- 2 .2 �� � + -.2 1 / ε ∈ R∗+2 / n ∈ N
∗>
supxn∈Xn
εp(Sxn) ≤ ε =⇒ N (ε,U , n) ≤ p
IJK LMNO P PQ R� S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � � � � � � � �� � � �� � � � � � � ��� �� � � � �� �
� � � � � � � � � � � � � � �� � � � � �� � � �� � �� �� �� �� � �
� � � � �- 2 .2 �� � � � �� � � . � � � � �2 %'& ( ( 6 *+ -.2 1 / E2 / F 82 : 2 : E 49 2 : 82 � 4 1 49 * 2 / S ∈ L (E,F )>
+ . S2 : / 82 ; 4 1 ) r D 4 7 -;: E - 0; n ∈ N∗ D
εn(S) ≤ 4‖S‖n−1/r
4 � , � � 5� � & & + -./ H7 4 3546 .7 72 82 : 3 -19 / . -1 : ; � 4 7 .: 4F 72 : E 4 ; 012 � C+ � � < Q9 4 / � ) -; .2 : : - 0:
7 @ * � E -/ * B : 2 � 02 Φ (X )2 : / . 19 7 0: 8 4 1: 7 4F - 072 82 ; 4 � -1 ΛΦ(X )9 2 1 / ; � 2 : 0; 7 @ -; . ) . 1 2 82 EΦ(X ) D � 02 72
= 2 9 / 2 0; w= � ; . �2 ‖w‖∞ ≤ Λw2 / � 02 b ∈ [−β, β]Q> + . 7 4 8.6 2 1: . -1 82 7 @2 : E 49 2 EΦ(X )2 : / � 1 .2 2 /
� ) 4 72 < d D 4 7 -;: D E - 0; / - 0/ γ ∈ R∗+ D
N (p) (γ/4,∆γH, 2m) ≤(
2
⌈
8β
γ
⌉
+ 1
)Q
·(
64ΛwΛΦ(X )
γ
)Qd
R(h) ≤ Rγ,m(h) +O
(
√
1
m
)
IJK LMNO P PQ R R S R)
� ��� M ��� �� � � L � � �JK M L �N � �
� � � � � � � � �� � � �� � � � � � � ��� �� � � � �� �
� � � � � � � � � � � � � � �� � � � � �� � � �� � �� �� �� � � �� � �
4 � , � � 5� � & � � 4 � , � � 5� �� � �� � � 7� � � �� % � � �� � � . � � � � � 2 %'& ( ( 6 *+ -.2 1 / H 01 2 : E 49 2 82
� .7F 2 ; / 2 / S 01 - E� ; 4 / 2 0; 4 E E 4 ; / 2 1 4 1 / < L (`n1 , H) - 0 L (H, `n∞)> 87 -;: D E - 0; / - 0/ 9 - 0 E72 8 @2 1 / .2 ;:
(k, n)= � ; . � 4 1 / 1 ≤ k ≤ n D -1 4ek(S) ≤ c
(
1
klog2
(
1 +n
k
)
)1/2
‖S‖,
-H 72 � " � L�� � � �! L � M �� �J� M �� � ek(S)2 : / � ) 4 7 < ε2k−1(S)2 / c2 : / 012 9 -1: / 4 1 / 2 01 .= 2 ;: 2 7 72 >
4 � , � � 5� � & + -./ H7 4 3546 .7 72 82 : 3 -19 / . -1 : ; � 4 7 .: 4F 72 : E 4 ; 012 � C+ � � < Q9 4 / � ) -; .2 : : - 0:
7 @ * � E -/ * B : 2 � 02 Φ (X )2 : / . 19 7 0: 8 4 1: 7 4F - 072 82 ; 4 � -1 ΛΦ(X )9 2 1 / ; � 2 : 0; 7 @ -; . ) . 1 2 82 EΦ(X ) D � 02 72
= 2 9 / 2 0; w= � ; . �2 ‖w‖∞ ≤ Λw2 / � 02 b ∈ [−β, β]Q> 87 -;: D E - 0; / - 0/ γ ∈ R
∗+ D
N (p)(γ/4,∆γH, 2m) ≤(
2
⌈
8β
γ
⌉
+ 1
)Q
· 216cΛwΛΦ(X)
γ
q
2Qm
ln(2)−1
R(h) ≤ Rγ,m(h) +O
(√
1√m
)
IJK LMNO P PQ R 9 S R)
� ���M ������
�L ���JKML�N ��
�������
����
������ ��� �
�����������������������������
E
H
C
?
<−−−−−−−−−−−−−−−−−−−−−−−−>
P(V in C | S)
...VKPVDNFDWSNYHGKWWEVAKYPNSVEKYGKCGWAE...
...VKPVDNFDWSNYHGKWWEVAKYPNSVEKYGKCGWAE...
NN 2 NN 3NN 1
SIMPA 96SOPMA
post−traitement (Si nécessaire)
Combinaison
P(V en H | S) P(V en E | S)
Réseaux de "filtrage"
M−SVM(GOR IV)
MLRC, M−SVM...
Programmation Dynamique (IHMM)
Contenu de la fenetre glissante S^
IJKLMNOPPQ
R)SR)