2005.12.27
Learning to Rank
Ming-Feng Tsai
National Taiwan University
2005.12.27
Ranking Ranking vs. Classification
Training samples is not independent, identical distributed. Criterion of training is not compatible to one of IR
Many ML approaches have been applied to ranking RankSVM
T. Joachims, SIGKDD, 2002 (SVM Light) RankBoost
Freund Y., Iyer, Journal of Machine Learning Research, 2003 RankNet
C.J.C. Burges, ICML, 2005 (MSN Search)
2005.12.27
Motivation RankNet
Pro Probabilistic ranking model Good properties
Con Training is not efficient Criterion of training is not compatible to one of IR
Motivation Based on the probabilistic ranking model Improve efficiency and loss function
2005.12.27
Probabilistic Ranking Model Probabilistic Ranking Model
Model posterior by Pij
The map from outputs to probabilities are modeled using a sigmoid function
Define ( ) and ( ) ( )
1
ij
ij
i i ij i j
o
ij o
o f x o f x f x
eP
e
( )i jP x x
Properties Combined Probabilities Consistency requirements
P(A>B)=0.5, and P(B>C)=0.5, then P(A>C)=0.5 Confidence, or lack of confidence, builds as expected.
P(A>B)=0.6, and P(B>C)=0.6, then P(A>C)>0.6
2005.12.27
Probabilistic Ranking Model
Cross entropy loss function Let be the desired target values
Total Cost Function:
RankNet applied this loss function by Nerural Network (BP network)
Applied this loss function by additive model
log(1 )ijo
ij ij ijC P o e
( ) log (1 ) log(1 )
log(1 )ij
ij ij ij ij ij ij
o
ij ij ij
C C o P P P P
C P o e
ijP
ijij
C
2005.12.27
Derivation of cross entropy loss function for Additive Model
1 1
( ) ( )
1
( ) ( ) ( ( ) ( ))
1 1
1, ,
log(1 )
( ( ) ( )) log(1 )
Let ( ) ( ) ( )
( ( ) ( ) ( ( ) ( )) log(1 )
Let
ij
i j
k i k j k k i k j
o
ij ij ijij ij
f x f x
ij i jij
k k k k
f x f x h x h x
ij k i k j k k i k jij
k i j
C P o e
P f x f x e
f x f x h x
P f x f x h x h x e
f
1, , , ,
1, , , ,
1, , , ,
1 1 , ,
1, , , ,
1, , , ,
,, ,
( ) ( ) and = ( ) ( )
( ) log(1 )
( ) log(1 )
k i j k k i j
k i j k k i j
k i j k k i j
k i k j k i j k i k j
f h
ij k i j k k i jij
f h
ij k i j k k i jij
f h
kij k i j
k
f x f x h h x h x
P f h e
J P f h e e
e e hJP h
1, , , ,
1, ,
,
, ,
2
1 11 1
01
Let , , ,
( ) 01
with some relaxations
( ) ( ( ) ( ) ) ( ) 01 1
k i j k k i j
k i j k
i j
f hij
f
ij k i j
b
b
b bb b
e e
a P b h c e x e
bcxab
cx
c cx a a x
c c
2005.12.27
Candidates of Loss Functions Cross entropy
KL-Divergence This loss function is equivalent to cross entropy
Information Radius KL-Divergence and cross entropy are asymmetric information radius is symmetric, that is, IRad(p,q)=IRad(q,p)
Minkowski norm
This seems simpler than cross entropy in mathematical derivation for boosting
( ; ) ( ) ( || ) ( ) log ( )x X
H X q H X D p q p x q x
( )( || ) ( ) log
( )x X
p xD p q p x
q x
( || ) ( || )2 2
p q p qD p D q
( , ) ( ) ( )x X
L p q p x q x
2005.12.27
Fidelity Loss Function Fidelity
A more reasonable loss function that is inspired from quantum computation
Hold the same properties in probabilistic ranking model proposed by Chris et al.
New properties F(p, q) = F(q, p) Loss is between 0 and 1 get the minimum loss value 0 the loss convergence
1 12 2
( , ) 1 ( ) ( )
11 ( ) (1 )*( )
1 1
ij
ij ij
x X
o
ij ij ijo o
F p q p x q x
eF P P
e e
2005.12.27
Fidelity Loss Function Properties
Total loss function
Pair-level loss is considerede.g. the loss of (5, 4, 3, 2, 1) and (4, 3, 2, 1, 0) is zero
Query-level loss is also considered
More penalty for larger grade of paire.g. (5, 0) and (5, 4)
q
1 1
| | | # of pairs | ijq ij
FQ
queyr1 query2 Loss
Case1 1000 0 0.5
case2 990 10 0.005
2005.12.27
Derivation for Additive Model
1
1 1
1 12 2
1 1( )
| | | # of pairs |
1 1 11 *( ) (1 )*( )
| | | # of pairs | 1 1
k ij k kij
k ij k kij k ij k kij
ijq ijq
f h
ij ijf h f hq ijq
J H FQ
eP P
Q e e
1( , )
| | | # of pairs |q
D i jQ
1
1 1
1 12 21
( ) ( , ) 1 *( ) (1 )*( )1 1
k ij k kij
k ij k kij k ij k kij
f h
ij ijf h f hij
eJ H D i j P P
e e
We denote
2005.12.27
Derivation for Additive Model
1 11
1 1 1 1
1 12 2
2 2
1 1 1( , ) *( ) * (1 )*( ) (1 )*
2 21 (1 ) 1 (1 )
k ij k kij k ij k kijk ij k kij
k ij k kij k ij k kij k ij k kij k ij k kij
f h f hf hkij kij
ij ij ij ijf h f h f h f hk
h e h eJ eD i j P P P P
e e e e
1 1
1 1
1 12 2
3 132 2
0
(1 )( , ) 0
( ) ( ( ) )
k ij k kij k ij k kij
k kij k ij k kij k kij k ij
ij
f h f hij ijkij kij
h f h h fij
h P e e h e P eD i j
e e e e e
1 1 1 1
1 1
1 1
1
1 11 12 22 2
3 31 12 2
1 12 2
2
(1 ) (1 )( , ) ( , )
(1 ) (1 )
(1 )( , )
(1
k ij k ij k ij k ij
k k
k ij k ijkij kij
k ij k ij
k
k
f f f fij ij ij ij
f fh h
f fij ij
f
P e e P P e e Pe D i j e D i j
e e
P e e PD i j
ee
1 1
1
31 2
1 12 2
31 2
,1
,1
)
(1 )( , )
(1 )
1ln
2
ijkij
k ij k ij
k ijkij
kij
kij
h
f fij ij
fh
i jh
ki j
h
P e e PD i j
e
W
W
2005.12.27
FRank
1 | |{(( , ), )}, ... , {(( , ), )}ij iji j Q i jq x x P q x x P
1( , )
| | | # of pairs |q
D i jQ
1
( ) ( )T
t tt
H x h x
Algorithm: FRankGiven: ground truth
Initialize:
For t=1,2, …, T(a) For each weak learner candidate hi(x)
(a.1) Compute optimal αt,i
(a.2) Compute the fidelity loss(b) Choose the weak learner ht,i(x) with the minimal loss as ht(x)
(c) Choose the corresponding αt,i as αt
(d) Update pair weight by Wi,j
Output the final ranking
2005.12.27
Implementation Finished
Threshold fast implementation Faster 4 times
Alpha fast implementation Faster 120 seconds per weak learner in 3w
Total loss fast implementation Faster 3 times
Resume to training Plan
Multi-Thread implementation Parallel Computation Margin Consideration (Fast, but with loss)
2005.12.27
Preliminary Experimental Results Data Set of BestRank
Competition Training Data: about 2,500
queries Validation Data: about
1,100 queries Features: 64 features
Evaluation NDCG
2005.12.27
Preliminary Experimental Results
Results of Validation Data
2005.12.27
Next step
2005.12.27
Interesting Analogy Loss function
Pair-level loss Query-level loss Other considerations
Learning Model Boosting, additive model LogitBoost Boosted Lasso SVM Neural Network
The whole new model The dependence of retrieved web pages
2005.12.27
Pairwise Pairwise Training
Ranking is reduced to a classification problem by using pairwise items as training samples
This increases the data complexity from O(n) to O(n2)
Suppose there are n samples evenly distributed on k ranks, the total number of pairwise samples is roughly n2/k
F(x,y)Xi
Xj
F(x)Xi
2005.12.27
Pairwise Pairwise Training
F(x,y) is more general function than F(x) – F(y) Find properties that should be modeled by F(x,y)
Nonlinear relation between x and y margin(r1, r30) > margin(r1, r10) > margin(r21, r30) …
2005.12.27
Pairwise Pairwise Testing
In testing phrase, rank should be reconstructed from a partial orders graph, even inconsistent and incomplete
Topological sorting can only handle DAG in linear time Problem
inconsistent How to find the best spanning tree
incomplete How to deal with the node without label
2005.12.27
Pairwise Spanning Tree Related Content
Colley’s Bias Free College Football Ranking Method
Tree Reconstruction via partial order
…
2005.12.27
Thanks your attention Q&A
2005.12.27
Additive Model AdaBoost
Construct a classifier H(x) by the linear combination of the base classifier h(x)
In order to obtain the optimal base classifiers {hT(x)} and linear combination coefficients {αT}, we need to minimize the training error
For binary classification problems (1 or -1), the training error for the classifier H(x) can be written as
11
( ) ( ) ( ) ( )T
t t T T Tt
H x h x H x h x
1
( ( ) ) /N
i ii
err sign H x y N
2005.12.27
Additive Model AdaBoost
For the simplicity of computation, it uses the exponential cost function as the objective function
Apparently, the exponential cost function upper bounds the training error err
1 1
( )
1
( ) ( )
1
1 21 2
1 2
1
1 { ( ( ), ) ( ( ), )}
where function I is defined as
1 ( , )
0
T i i
T i i T i i
NH x y
i
NH x y H x yT T
T i i T i ii
err eN
e e I h x y e e I h x yN
if x xI x x
if x x
2005.12.27
Additive Model AdaBoost
By setting the derivative of the equation above with respect to αT to be zero, we have the expression as follows:
1
1
( )
1
( )
1
( ( ), )1ln
2 ( ( ), )
T i i
T i i
N H x yT i ii
T N H x yT i ii
e I h x y
e I h x y
With the expression of data distribution1
1
( )
( )
1
T i i
T j j
H x yT
i N H x y
j
eW
e
the linear combination coefficient αT can be written as
1
1
( ( ), )1 1 1ln ln( )
2 2( ( ), )
where stand for the weighted error rate under the weight distribution
for the base classifier ( ) in iteration T
N T Ti T i ii
T N TTi T i ii
T T
T
W I h x y
W I h x y
W
h x
2005.12.27
Additive ModelGiven: (xi, yi)1. Initialize: W1=1/N2. For t=1,2, …, T
(a) Train weak learner using distribution Wt
(b) compute
(c) compute
(d) update
3. Output
1( ( ), )
Nt ti t i ii
W I h x y
1 1
ln( )2
t
t t
1
( ) ( )T
t tt
H x sign h x
( )i t i ty h xt ti iW W e
Back
2005.12.27
NDCG K Jaervelin, J Kekaelaeinen - ACM Transactions on
Information Systems, 2002 Example
Assume that the relevance scores 0 – 3 are used.
G’=<3, 2, 3, 0, 0, 1, 2, 2, 3, 0, …> Cumulated Gain (CG)
CG’=<3, 5, 8, 8, 8, 9, 11, 13, 16, 16, …>
[1], if 1 [ ]
[ 1] [ ], otherwise
G iCG i
CG i G i
2005.12.27
NDCG Discount Cumulated Gain (DCG)
let b=2,
DCG’=<3, 5, 6.89, 6.89, 6.89, 7.28, 7.99, 8.66, 9.61, 9.61, …> Normalized Discount Cumulated Gain (NDCG)
Ideal vector
I’=<3, 3, 3, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, …>
CGI’=<3, 6, 9, 11, 13, 15, 16, 17, 18, 19, 19, 19, 19, …>
DCGI’=<3, 6, 7.89, 8.89, 9.75, 10.52, 10.88, 11.21, 11.53, 11.83, 11.83, …>
NDCG’=<1, 0.83, 0.89, 0.73, 0.62, 0.6, 0.69, 0.76, 0.89, 0.84, …>
[1], if 1 [ ]
[ 1] [ ] / log , otherwiseb
G iDCG i
DCG i G i i
Back