Tamas Jambor, Jun Wang, Neal Lathia(@jamborta,@seawan,@neal_lathia)

University College London

Recommender systems exists in time. Manyaspects of the system can only be

understoodover time.

Understanding and controlling dynamic aspects of the systems to gain control over certain objectives.

Predictability over time Allocating resources over time Stable system over time

Describe and estimate the system dynamics Build and analyze a feedback loop controller Test the dynamics with real data

More data -> better performance Frequent training -> Up-to-date

recommendation

Performance loss over time

Use all the data -> best performance Use part of the data -> frequent and regular

updates

Relationship between the number of training samples (computation) and performance.

How certain input now would change the future state of the system.

y = displacementF = force appliedm = mass of the blockr = damping constantk = spring constant

k

r

F(t)

y(t)

m

Using Newton’s second law

◦ Spring force:◦ Damping force:

This gives the equation

)('')( tmytF

)(1 tkyf )('2 tryf

)(')()( trytkytF

k

r

F(t)

y(t)

m

y = performance F = number of training samplesk = constant related to the new training samplesr = constant related to the change in the new training samples

k

r

F(t)

y(t)

)(')()( trytkytF

Generate an input signal Log-normal random walk to

add noise Maximum likelihood

estimation

Build a control loop based on system dynamics

Actuator

Monitor

Reference performance

control input

Performance variable

Manipulated Variable (e.g., number of

new training samples)

Controlled Recommender System

+ -

error

ControlFunction

Controller

Dynamics

PerformanceDynamics

Controller)(tu

)(tn

Disturbance

)(te

Reference Value

)(ty

)(tyr

+

-

)()()( tytyte r

)(tyRec.

Accuracy

Measurement

Control Signal

Rec. Error

Rise time Settling time Overshoot

Type Rise time (sec)

Settling time (sec)

Overshoot (%)

P 5 10 0

PD 1 5 8.21

PI 0 10 36.8

PID 0 10 15.7

P controller PD controller PI controller PID controller

How the controller can perform with unseen data.

Simulate system for 30 days (once a day) 3 reference values (RMSE 0.95, 0.9, 0.86) Measure

◦ Settling time◦ Error (with respect to reference) – RMSE-SS◦ Stability (with respect to reference) – SD-SS◦ Overshoot

PD controller

Type Settling time

RMSE-SS (error)

SD-SS (stability)

Overshoot (%)

P (0.86) 23 0.0104 0.0034 0

PD (0.86) 20.8 0.0059 0.0037 0.18

PI (0.86) 18.6 0.0055 0.0047 0.49

PID (0.86) 18.6 0.0045 0.0045 0.72

Computation Computational gainPerformance loss

Fixed computational cost would also fix update frequency (given the resources available)

System can provide regular updates independent from the rate of the new samples

Resources are used to their limits (e.g. EC2)

RECOMMENDATION SERVER

RECOMMENDER INPUT CONTROLLER

USAGE DATA PROCESSOR

TRAINING DATAPROCESSOR

RECOMMENDATION ASSESSOR

feedback

request data

recommendation

usage data

Certain aspects of the system are to be controlled

Stability and predictability are important Use resources to their limits

Optimise rather than stabilise over time Obtain system dynamics Describe objective to be achieved (finite or

infinite horizon)

Describe system dynamics Design a feedback loop Evaluate control system on real data

Thank you.