Date post: | 10-May-2015 |
Category: |
Education |
Upload: | simon-dooms |
View: | 450 times |
Download: | 1 times |
A User-centric Evaluation of Recommender Algorithms for an Event Recommendation SystemSimon Dooms, Toon De Pessemier, Luc Martens
@sidooms
Introduction
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 2
• Online context• Cultural events• 30,000 items
Recommender System?
Concl.ResultsQuestionsAlgorithmsFeedbackExperimentIntroIntro
The Experiment
Invitation mailDAY 1
Closed questionnaireDAY 56
DAY 41
End tracking
DAY 45
Send out recs
DAY 50Reminder mail
Reminder mailDAY 28
Concl.ResultsQuestionsAlgorithmsFeedbackIntroExperimentExperiment
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 3
FeedbackConcl.ResultsQuestionsAlgorithms
FeedbackFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 4
Algorithms
Content-Based CBUser-Based Collaborative Filtering UBCFHybrid recommender CB+UBCFSingular Value Decomposition SVDRandom recommender RAND
Concl.ResultsQuestionsAlgorithmsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 5
QuestionnaireConcl.Results
QuestionsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 6
AccuracyFamiliarity
NoveltyDiversity
TransparencySatisfaction
TrustUsefulness
Results
612 users
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 7
232 users193 users
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
Average scores per question and algorithm
Accuracy Familiarity Novelty Diversity Trans. Satisfaction Trust Usefulness10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 8
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
Average scores per question and algorithm
Accuracy Satisfaction910/23/2011 Simon Dooms - Ghent University - UCERSTI 2
CorrelationsConcl.
ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 10
AccuracyFamiliarity
NoveltyDiversity
TransparencySatisfaction
TrustUsefulness
RegressionsConcl.
ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 11
Conclusions
• Online user-centric evaluation experiment• 5 popular algorithms: CB, UBCF, CB+UBCF, SVD, RAND
• CB+UBCF winner algorithm• SVD ‘loser’ algorithm• Accuracy and Transparency Satisfaction• Diversity NOT correlated with Satisfaction
Concl.Concl.ResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 12
Simon Dooms, Toon De Pessemier, Luc Martens
A User-centric Evaluationof Recommender Algorithms for an Event Recommendation System
With the support of IWT Vlaanderen and FWO Vlaanderen