Inductive Framework for Multi-Aspect ... - Madhav Nimishakavi · Madhav Nimishakavi CIKM 2018....

IntroductionPreliminaries

Side Information infused Incremental Tensor Analysis (SIITA)Results

Inductive Framework for Multi-Aspect StreamingTensor Completion with Side Information

Madhav Nimishakavi1 Bamdev Mishra2 Manish Gupta2

Partha Talukdar1

1Indian Institute of Science 2Microsoft

Madhav Nimishakavi CIKM 2018



Outline

1 Introduction

2 Preliminaries

3 Side Information infused Incremental Tensor Analysis (SIITA)

4 Results




Introduction

A Tensor is a multi-way extension of a matrix.

Month/Year

Users

Movies

r

rating

Tensors are used for representing multidimensional data.




Introduction (cont.)

In practice, many multidimensional datasets are oftenincomplete.

Tensor Completion is the task of predicting or imputingmissing values in a partially observed tensor.





However, in many real world applications the data is dynamic.Some examples include,

Online recommendation systems.

Social networks.

. . .

Dynamic Tensor Completion is the task of predictingmissing values in a dynamically growing partially observedtensor.





Most of the existing works make an assumption that thetensor grows only in one mode.

observed at the previoustime stamps

observed at the currenttime stamp

t t+1 t+2

Figure : Streaming tensor sequence

This assumption is restrictive !





Recently Song et al. [4] proposed the more generalMulti-aspect streaming tensor completion.



t t+1 t+2

Figure : Multi-aspect streaming tensor sequence





Besides the tensor, additional side information data is alsoavailable in the form of matrices in many applications.

For example, movie × genre matrix for online movierecommendation etc.

Incorporating the side information matrices into tensorcompletion can help achieve better results, particularly insparse settings.





We propose a framework to handle the following sequences.

t t+1 t+2

(a) Streaming sequence with side

information

t t+1 t+2



side information matrix

(b) Multi-aspect streaming sequence

with side information




Preliminaries

Definition (Multi-aspect streaming Tensor Sequence) [4]: Atensor sequence of Nth-order tensors X (t) is called amulti-aspect streaming tensor sequence if for any t ∈ Z+,

X (t−1) ∈ RI t−11 ×I t−1

2 ×...×I t−1N is the sub-tensor of

X (t) ∈ RI t1×I t2×...×I tN , i.e.,

X (t−1) ⊆ X (t), where I t−1i ≤ I ti , ∀1 ≤ i ≤ N.

Here, t increases with time, and X (t) is the snapshot tensor of thissequence at time t.




Preliminaries (cont.)

Definition (Multi-aspect streaming Tensor Sequence with

Side Information) : Given a time instance t, let A(t)i ∈ RI ti ×Mi be

a side information (SI) matrix corresponding to the i th mode ofX (t), we have,

A(t)i =

[A

(t−1)i

∆(t)i

], where ∆

(t)i ∈ R[I

(t)i −I

(t−1)i ]×Mi .

let the side information set A(t) = A(t)1 , . . . ,A

(t)N .

Given an Nth-order multi-aspect streaming tensor sequenceX (t), we define a multi-aspect streaming tensor sequence withside information as (X (t),A(t)).




Preliminaries (cont.)

Problem Definition: Given a multi-aspect streaming tensorsequence with side information (X (t),A(t)), the goal is topredict the missing values in X (t) by utilizing only entries in therelative complement X (t) \X (t−1) and the available sideinformation A(t).




SIITA

We propose Side Information infused Incremental TensorAnalysis (SIITA).

Property TeCPSGD[3] OLSTEC[2] MAST[4] AirCP[1] SIITA

Streaming X X X XMulti-Aspect Streaming X XSide Information X XSparse Solution X

Table : Summary of different tensor streaming algorithms.




SIITA (cont.)

minG∈Rr1×...×rN

Ui∈RMi×ri ,i=1:N

F (X (t),A(t),G, Uii=1:N), (1)

where

F (X (t),A(t),G, Uni=1:N) =∥∥∥∥∥∥∥observed tensor at t︷︸︸︷PΩ(X (t))−PΩ(G ×1

side info at t︷︸︸︷A

(t)1 U1 ×2 . . .×N

side info at t︷︸︸︷A

(t)N UN)

∥∥∥∥∥∥∥2

F

+ λg ‖G‖2F +

N∑i=1

λi ‖Ui‖2F . (2)




SIITA (cont.)

Since (X (t−1),A(t−1)) ⊆ (X (t),A(t)) , we have

F (X (t),A(t),G(t−1), U(t−1)i i=1:N) =

term at time t − 1︷︸︸︷F (X (t−1),A(t−1),G(t−1), U(t−1)

i i=1:N) +

F (X (∆t),A(∆t),G(t−1), U(t−1)i i=1:N)︸︷︷︸

delta term between t and t − 1

(3)




SIITA (cont.)

We propose the following incremental update scheme, U(t)i = U

(t−1)i − γ ∂F (∆t)

∂U(t−1)i

, i = 1 : N

G(t) = G(t−1) − γ ∂F (∆t)

∂G(t−1) ,

SGD style updates

where γ is the step size for the gradients. R(∆t), needed forcomputing the gradients of F (∆t), is given by

R(∆t) = X (∆t) − G(t−1) ×1 A(∆t)1 U

(t−1)1 ×2 . . .

×NA(∆t)N U

(t−1)N .

(4)




SIITA (cont.)

Algorithm 1: Proposed SIITA Algorithm

Input : X (t),A(t), λi , i = 1 : N, (r1, . . . , rN )

Randomly initialize U(0)i ∈ RMi×ri , i = 1 : N and G(0) ∈ Rri×...×rN ;

for t = 1, 2, . . . do

U(t)0i

:= U(t−1)i , i = 1 : N;

G(t)0 := G(t−1);for k = 1:K do

Inner iterations

Compute R(∆t) from (4) using U(t)k−1i , i = 1 : N and G(t)k−1 ;

Compute ∂F (∆t)

∂U(t)k−1i

for i = 1 : N ;

Update U(t)ki using ∂F (∆t)

∂U(t)k−1i

and U(t)k−1i ; Updating Factor Matrices

Compute ∂F (∆t)

∂G(t)k−1;

Update G(t)k using G(t)k−1 and ∂F (∆t)

∂G(t)k−1; Updating Core Tensor

end

U(t)i

:= U(t)Ki ; G(t) := G(t)K ;

end

Return: U(t)i , i = 1 : N,G(t).




Results

MovieLens 100K YELP

Modes user × movie × week user × business × year-month

Tensor Size 943×1682×31 1000×992×93

Starting size 19×34×2 20×20×2

Increment step 19, 34, 1 20, 20, 2

Sideinfo matrix 1682 (movie) × 19 (genre) 992 (business) × 56 (city)

Table : Summary of datasets used in the paper.




Multi-Aspect Streaming Setting

0 10 20 30 40 500

0.5

1

1.5

2

2.5

3

Time Step

Test

RM

SE

SIITAMAST

(a) MovieLens 100K

(20% Missing)

0 10 20 30 40 500

0.5

1

1.5

2

2.5

3

3.5

4

Time Step

Test

RM

SE

(b) YELP

(20% Missing)

Figure : Evolution of Test RMSE (lower is better) of MAST and SIITAwith each time step.




Multi-Aspect Streaming Setting (cont.)

0 10 20 30 40 500

500

1000

1500

2000

2500

3000

3500

4000

4500

Time Step

Ru

n T

ime (

seco

nd

s)

SIITAMAST

(a) MovieLens 100K

(20% Missing)

0 10 20 30 40 500

500

1000

1500

2000

2500

3000

Time Step

Ru

n T

ime (

seco

nd

s)

(b) YELP

(20% Missing)

Figure : Runtime comparison between MAST and SIITA at every timestep.




Streaming Setting

0 5 10 15 20 25 301

2

3

4

5

6

7

8

Time Step

Tes

t R

MS

E

SIITAOLSTECTeCPSGD

(b) MovieLens 100K

(20% Missing)

0 20 40 60 801

1.5

2

2.5

3

3.5

4

4.5

5

Time Step

Test

RM

SE

(a) YELP

(20% Missing)

Figure : Evolution of Test RMSE (lower is better) of TeCPSGD,OLSTEC and SIITA with each time step.




Streaming Setting (cont.)

0 5 10 15 20 25 300.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

Time Step

Ru

n T

ime (

seco

nd

s)

SIITAOLSTECTeCPSGD

(b) MovieLens 100K

(20% Missing)

0 10 20 30 40 50 60 70 80 900

2

4

6

8

10

12

14

16

Time Step

Ru

n T

ime (

seco

nd

s)

(a) YELP

(20% Missing)

Figure : Runtime comparison between TeCPSGD, OLSTEC and SIITA.




Static Setting

Dataset Missing% Rank AirCP SIITA

MovieLens100K

20%3 3.351 1.5345 3.687 1.67810 3.797 2.791

50%3 3.303 1.5805 3.711 1.58510 3.894 2.449

80%3 3.883 1.5545 3.997 1.65410 3.791 3.979

Table : Mean Test RMSE (lower is better) across multiple train-testsplits in the Batch setting.




Static Setting (cont.)

Dataset Missing% Rank AirCP SIITA

YELP

20%3 1.094 1.0525 1.086 1.05610 1.077 1.181

50%3 1.096 1.0975 1.095 1.05910 1.719 1.599

80%3 1.219 1.1995 1.118 1.15610 2.210 2.153

Table : Mean Test RMSE (lower is better) across multiple train-testsplits in the Batch setting.




Nonnegative Setting

Incorporating Nonnegative constraints into SIITA(NN-SIITA) is useful for unsupervised setting.

Metrics for evaluating the clusters mined by NN-SIITA

Let wp items of top w items in a cluster belong to the samecategory, then

For a cluster p,Purity(p) = wp/w ,

average-Purity =1

ri

ri∑p=1

Purity(p),

where ri is the number of clusters along mode-i .




Nonnegative Setting (cont.)

0 10 20 30 40 50

0.5

0.6

0.7

0.8

0.9

Time Step

Avera

ge P

uri

ty

NN−SIITANN−SIITA (w/o SI)

(a) MovieLens 100K

0 10 20 30 40 50

0.4

0.5

0.6

0.7

0.8

0.9

1

Time Step

Avera

ge P

uri

ty

(b) YELP

Figure : Average Purity (higher is better) of clusters learned by NN-SIITAand NN-SIITA (w/o SI) at every time step in the unsupervised setting.




Nonnegative Setting (cont.)

5 10 15 20 250.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

w

Mea

n a

vera

ge p

uri

ty

NN−SIITANN−SIITA (w/o SI)

(a) MovieLens 100K

5 10 15 20 250.45

0.5

0.55

0.6

0.65

0.7

w

Mean

avera

ge p

uri

ty

(b) YELP

Figure : Evolution of mean average purity (higher is better) with w forNN-SIITA and NN-SIITA (w/o SI) for both MovieLens 100K and YELPdatasets.




Takeaways

SIITA is the first ever algorithm that incorporates sideinformation into dynamic tensor completion.

SIITA can handle the more general Multi-aspect streamingsetting.

NN-SIITA is the first ever algorithm that incorporatesNonnegative constraints into dynamic tensor analysis.

Codes available at https://madhavcsa.github.io


https://madhavcsa.github.io



Thank You!




Bibliography

[1] H. Ge, J. Caverlee, N. Zhang, and A. Squicciarini.Uncovering the spatio-temporal dynamics of memes in thepresence of incomplete information.CIKM, 2016.

[2] H. Kasai.Online low-rank tensor subspace tracking from incomplete databy cp decomposition using recursive least squares.In ICASSP, 2016.

[3] M. Mardani, G. Mateos, and G. B. Giannakis.Subspace learning and imputation for streaming big datamatrices and tensors.IEEE Transactions on Signal Processing, 2015.




Bibliography (cont.)

[4] Q. Song, X. Huang, H. Ge, J. Caverlee, and X. Hu.Multi-aspect streaming tensor completion.In KDD, 2017.


Date post:	05-Aug-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Inductive Framework for Multi-Aspect ... - Madhav Nimishakavi · Madhav Nimishakavi CIKM 2018....

Documents