1
CFM:ConvolutionalFactorizationMachinesforContext-Aware
Recommendation
XinXin, BoChen,Xiangnan He,etal.
School of ComputingScience,UniversityofGlasgowSchoolofSoftwareEngineering,ShanghaiJiaotongUniversity
SchoolofDataScience,UniversityofScienceandTechnologyofChina
1 2 3
1
2
PresentedbyXinXin@IJCAI19,Aug.16,2019
3
FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms
2
FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms
• One/Multi-hotfeaturevectorsasinputs– Encodesbothitem/usersideinformationandcontextinformation
3
FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms
• One/Multi-hotfeaturevectorsasinputs• Combineslinearregressionandsecond-orderfeatureinteraction
4
linearregression second-orderfeatureinteraction
feature embeddingfor
LimitationsofFM• Innerproductbasedfeatureinteraction– Embeddingdimensionsareindependentwitheachother
– Theremaybecorrelationsbetweendifferentdimensions[Zhangetal.,SIGIR2014]
• Higher-orderinteraction&Non-linearity– NFM[Heetal.,SIGIR2017]– DeepFM [Guo etal.,IJCAI2017]
5
Innerproduct ?
Contributions• Utilizeanouterproduct-basedinteractioncubetorepresentfeatureinteractions,whichencodesbothinteractionsignalsanddimensioncorrelations.
• Employ3DCNNabovetheinteractioncubetocapturehigh-orderinteractionsinanexplicitway.
• Leverageanattentionmechanismtoperformfeaturepooling,reducingtimecomplexity.
6
ConvolutionalFactorizationMachines(CFM)
• Predictionrule:
• Overallstructure:
7
CFM• InputandEmbeddingLayer– sparsefeaturevectors==>embeddingtablelookup
• Attentionpoolinglayer
8
CFM• InputandEmbeddingLayer– sparsefeaturevectors==>embeddingtablelookup
• Attentionpoolinglayer
– attentionscore
– softmax
– weightedsum
9
CFM• InteractionCube
• 3DCNN
10
CFM• ModelTraining– Pair-wiserankingloss(BPR)[Rendle etal.,UAI2009]
– L2regularization– Drop-out
11
Experiments• Researchquestions:– DoesCFMmodeloutperformstate-of-the-artmethodsfortop-k recommendation?
– HowdothespecialdesignsofCFM(i.e.,interactioncubeand3DCNN)affectthemodelperformance?
– What’stheeffectoftheattention-basedfeaturepooling?• Datasets:– Frappe– Last.fm– MovieLens
• Evaluation:– Leave-one-out– HR&NDCG
12
Experiments• Baselines:– PopRank:popularity-basedrecommendation– FM[Rendle etal.,ICDM2010]:originalFMwithBPRloss– NFM[Heetal.,SIGIR17]:stackingMLPuponFM– DeepFM[Guo etal.,IJCAI2017]:wide&deep+FM– ONCF[Heetal.,IJCAI2018]:outerproduct+MF
13
Experiments• RQ1(performance)
– DeepstructurehelpstoimproveFM(DeepFM&NFM)– CFMachievesthebestperformance
Frappe PopRank FM DeepFM NFM ONCF CFM
HR@10 0.3493 0.5486 0.6035 0.6197 0.6531 0.6720
NDCG@10 0.1898 0.3469 0.3765 0.3924 0.4320 0.4560
Last.fm PopRank FM DeepFM NFM ONCF CFM
HR@10 0.0023 0.2382 0.2612 0.2676 0.3208 0.3538
NDCG@10 0.0011 0.1374 0.1473 0.1488 0.1823 0.1948
Frappe PopRank FM DeepFM NFM ONCF CFM
HR@10 0.0235 0.0998 0.1170 0.1192 0.1110 0.1323
NDCG@10 0.0107 0.0452 0.0526 0.0553 0.0514 0.0627
14
Results• RQ2(modelablation)– Interactioncube&3DCNN
15
3Darchitecturehelpstoimproveperformance
Results• RQ3(featurepooling)– Effectofattention
– Runtime&performance
16
Attentionpooinglayerhelpstoimprovebothefficiencyandeffectiveness
Conclusion&FutureWork• CFMforfeature-basedrecommendation– Outer product-basedinteractioncube– 3DCNNtoexplicitlylearnhigh-orderinteractions– Attention-basedfeaturepoolinglayertoreducecomputationalcost
• Futurework– Improveefficiency– Residuallearning
17
Reference• [Rendle etal.,2010]Factorizationmachines.InICDM.• [Rendle etal.,2009]Bpr:Bayesianpersonalizedrankingfrom
implicitfeedback.InUAI.• [He,etal.,2017]Neuralfactorizationmachinesforsparse
predictiveanalytics.InSIGIR.• [Guo etal.,2017] Deepfm:A factorization-machinebased
neuralnetworkforctr prediction.InIJCAI.• [Heetal.,2018]Outerproduct-basedneuralcollaborative
filtering.InIJCAI.• [Zhangetal.,2014]Explicitfactormodelsforexplainable
recommendationbasedonphrase-levelsentimentanalysis.InSIGIR.
18
ThankyouQ&A
19
