CoFiRank : Maximum Margin Matrix Factorization for Collabora
tive Ranking
Markus Weimer, Alexandros Karatzoglou, Quoc Viet Le and Alex Smola
NIPS’07
Idea
• Maximum Margin Matrix Factorization
• Structured Estimation for Ranking
• Bundle Method Solver
Collaborative Filtering
• Based on partial observed matrix to predict unobserved entries
Matrix Factorization
• Low Rank Approximation
• SVD for fully observed Y
• Non-convex
Maximum Margin Matrix Factorization
• Trace norm+Hinge loss: Convex
• Semi-Definite Programming
Regularized Matrix Factorization
• Formulation
• Probabilistic Matrix Factorization (PMF)
• CoFiRank
• Linear Convex Upper Bound
TTtrain
U,VVVUUYUVL trtr
2),( minimize
2
2),( traintrain YUVYUVL
),NDCG(1),( traintrain YY Non-Convex
ffcYYfL traintrain ,),(max:),(
Solved by linear programming
Alternating optimizing
How to Compute Loss?
• Linear Convex Upper Bound
• Solved by Linear Programming
ffcYYfL traintrain ,),(max:),(
i j jijiX
XC ,, min
Can this explain in simple way?
Useful Links
• CoFiRank
http://www.cofirank.org
• MMMF http://ttic.uchicago.edu/~nati/mmmf/
• MFhttp://helikoid.si/mf/index.html
Famous Researchers in Optimization
• Yurii Nesterov – “Introductory Lectures on Convex Optimization: A Basic Course”http://www.core.ucl.ac.be/~nesterov/
• Arkadi Nemirovski – “Efficient methods in convex programming” http://www2.isye.gatech.edu/~nemirovs/
• Stephen P. Boyd – “Convex Optimization”http://www.stanford.edu/~boyd/
• Stephen J. Wright – “Numerical Optimization” http://pages.cs.wisc.edu/~swright/
• Dimitri Bertsekas – “Nonlinear Programming” http://web.mit.edu/dimitrib/www/home.html
Questions?
Normalized Discounted Cumulative Gain (NDCG)
How to set c?
• ci is set decreasing, is maximized with respect toπ for argsort(f)
• ci =(i+1)-0.25
fcf ,:),(
Convex Upper Bound
• Linear Convex Upper Bound
ffcYYfL traintrain ,),(max:),(
),(,),(),( **traintraintrain YffcYYfL
baba ,)(sort),(sort
Bundle Method
• General convex optimization solver with tight convergence bound O(1/)
Bundle Method for CoFiRank