Latent Dirichlet Allocation and Its Application in Recommder Systems

Weike Pan

Thanks to Ms. Qing Zhang

College of Computer Science and Software Engineering, Shenzhen University

Introduction

Topic modeling

Introduction

Token vs. term

• Note that word-instance = token and term = word.

• For example, in a document “my name is peter and my nationality is usa”, there are two tokens of the term “my”.

Introduction

Notations

Modeling

Graphical model

Modeling

Generation

Modeling

Objective function

Approximate Inference

• Exact inference

• Approximate inference

– Variational method

– Collapsed Gibbs sampling (we adopt this approach in this slides)

• Collapsed Gibbs Sampling

– A Markov chain Monte Carlo (MCMC) algorithm

– Main idea:

• For the current token w

• Calculate the probability that w belongs to each topic

• Sample a topic according to the probability

Algorithm

Algorithm (Collapsed Gibbs Sampling)

Application in Recommender Systems

• In recommender systems (in particular of one-class collaborative filtering), we may take users as documents, and items as terms, and model the users’ behaviors using LDA.

• Notice that the algorithm in previous pages can be used without modification.

• ...

References

• David M. Blei, Andrew Y. Ng and Michael I. Jordan. Latent Dirichlet Allocation. JMLR 2003.

• Thomas L. Griffiths and Mark Steyvers. Finding Scientific Topics. PNAS 2004.

• David M. Blei. Probabilistic Topic Models. CACM 2012.

• Haijun Zhang, Zhoujun Li, Yan Chen, Xiaoming Zhang and Senzhang Wang. Exploit Latent Dirichlet Allocation for One-Class Collaborative Filtering. CIKM 2014.