The Michelin Curse: Expert Judgment Versus Public … Michelin Curse: Expert Judgment Versus Public...

The Michelin Curse: Expert Judgment Versus Public Opinion

The Michelin Curse: Expert Judgment VersusPublic Opinion

Marco Del [email protected]

The University of Warwick

ICUR 2017Day 2

mailto:[email protected]


Outline

1 Introduction

2 The Data

3 Methodology and Hypotheses

4 Results

5 Conclusion


Introduction

Outline

1 Introduction

2 The Data


4 Results

5 Conclusion


Introduction

Context

Context

When choosing what restaurant to go to, what movie to watch,what book to read, or what wine to buy, observable productcharacteristics are unlikely to provide a complete picture of whatto expect.

Thus, the average consumer might rely on external sources ofinformation to make more informed decisions about a particulargood or service.

Without loss of generality, these sources might take two forms:expert judgment and public opinion.


Introduction

The Michelin Guide

The Michelin Guide

Sometimes referred to as “the foodies’ bible”, the Michelin Guideis considered to be the foremost judge of restaurant excellence.

Every year since 1926, the French company Michelin publishesa set of guides by geographical location, where it unveils whichrestaurants around the world have gained (or lost)

1 (“A very good restaurant in its category”)

2 (“Excellent cooking, worth a detour”)

3 (“Exceptional cuisine, worth a special journey”)

Michelin star(s) 1.

1Or as the French like to call them macarons, not to be confused withthe English word macaroons


Introduction

TripAdvisor

TripAdvisor

It has been argued that the most prominent user-generated re-view site within the travel industry is TripAdvisor.com.

In fact, TripAdvisor.com is currently visited on average, by 350million unique users monthly. In 2017, it reached 385 millionreviews covering 6.6 million accommodations, restaurants andattractions.


Introduction

Research Questions

Research Questions

By using the Michelin Guide Main Cities of Europe 2016 as thesource of expert opinion and TripAdvisor user-generated content(UGC) as the source of electronic word-of-mouth (eWOM) in therestaurant industry, we seek to answer three questions:

1 To what extent does expert judgment, in the form of Miche-lin stars, and public opinion, in the form of TripAdvisoruser-generated content, differ in the context of restaurants?

2 What is the most important background information abouta TripAdvisor reviewer in explaining the possible differencesbetween eWOM and expert opinion?

3 Given two restaurants a and b where a has one Michelin starand b does not have any but shares the same price range,cuisine, and city, is the average TripAdvisor overall ratingassociated with a lower than the one associated with b?


The Data

Outline

1 Introduction

2 The Data


4 Results

5 Conclusion


The Data

An Overview

An Overview

Restaurants (703,305): This dataset contains information aboutall the restaurants taken into consideration in theanalysis. Namely, it includes information about theMichelin restaurants in the Michelin Guide MainCities of Europe 2016 (482 in total), the restaurantsin the cities featured in the Guide.

Users (142,942): This dataset contains information aboutthe users who have given at least one review to arestaurant featured in the Guide.

Reviews (211,609): This dataset contains information aboutthe reviews that have been given to the restaurantsfeatured in the Guide.

Visits (2,947,227): This dataset contains information re-garding which user (in Users) visited which restau-rant (in Restaurants).


The Data

An Overview

An Overview

Figure: Geolocation of the users


The Data

Data Exploration: Restaurants Network Visualisation


Data visualisation methodology:

1 Create an undirected graph where each node is a restaurantand two nodes are joined by a weighted edge if and only if atleast one person has been to both, where the weight is givenby the number of people for which this holds.

2 Colour the nodes by number of Michelin stars and colour theedges by mixing the source and target colours.

3 Let the size of the nodes be given by the correspondingelement in the equilibrium distribution of the underlyingMarkov Chain as given by the PageRank algorithm.


The Data



Figure: 1 Michelin Star, 2 Michelin Star, 3 Michelin Star.


The Data



Figure: Node colour given by modularity class.


The Data

Storage

Storage

In order to establish rela-tional links between the datasetswhich would benefit data con-sistency and retrieval, a MySqlrelational database consistingof four tables (Restaurants,Users, Reviews and Visits)was created.

Figure: Database schema.


The Data

Collection

Collection

For the sake of rapid prototyping and re-usability, a web-scraperin R was implemented.

The library utilised to communicate with the remote web driverwas RSelenium, and the chosen browser was PahntomJS (a scripted,headless browser for automating web page interaction).

The web-scraper collected the data between September 15, 2016and February 25, 2017 whilst running on a Red Hat server.


Methodology and Hypotheses

Outline

1 Introduction

2 The Data


4 Results

5 Conclusion



Hypothesis 1

Hypothesis 1

Hypothesis 1 : When comparing the mean value of TripAdvisorratings between one, two, and three star(s) Michelinrestaurants, one should not observe any significantdifference.

Methodology :1 All the ratings were recomputed using the re-

views, so to be able to work with non- roundedfigures (Figure on next slide).

2 A non-parametric Wilcoxon rank sum test wasconducted to asses whether the location of theratings distributions changes between 1-2, 1-3, and 2-3 Michelin Stars with the followinghypothesis:

H0 true location shift is equal to 0H1 true location shift is not equal to 0.



Hypothesis 2

Hypothesis 2

Hypothesis 2 : Information about online reviewers in the formof demographics, restaurants visits, level of activityon the online platform, and trustworthiness can helpto explain as well as predict the possible differencesbetween eWOM, in terms of overall rating given to arestaurant, and expert opinion in terms of Michelinstars.

Methodology : The following models were trained on a (wide)panel data set.

1 Multinomial Logit model (MLogit): used toasses the marginal effects of the user backgroundinformation on the rating difference.

2 Multilayer Perceptron (MLP): used to gaugehow well the difference in the ratings could bepredicted.



Hypothesis 2

Hypothesis 2

We define the difference between the number of Michelin starsand the TripAdvisor rating as

di = φ(ri −mj)

where ri is the overall rating ri left in review i for restaurantj mapped onto the interval [1, 3], mj is the number of Michelinstars that restaurant j has, and φ is the function

φ(x) =

very positive x > 1.5

positive 0 ≤ x ≤ 1.5

negative x < 0

,



Hypothesis 2

Hypothesis 2

Table: Panel dataset variables

Variable name Description

user sexThe gender of the user: female (1),man (0)

user number of visits one starsThe number of times that a specific usershas visited one Michelin star restaurants.

user number of visits two starsThe number of times that a specific usershas visited two Michelin star restaurants

user number of visits three starsThe number of times that a specific usershas visited three Michelin star restaurants

user meanCategoryPriceThe mean price category of all therestaurants visited by the user

user mean review sentiment

The mean text polarity sentimentexpressed by the reviews left by the user

on a continuous scale form + to - 2

user level The level of the useruser hometown lon The longitude of the user hometownuser hometown lat The latitude of the user hometown

number of helpful votes michelin restaurantsThe total number of helpful votes givento the reviews left by the user

rating difference φ(ri −mj)



Hypothesis 3

Hypothesis 3

Hypothesis 3 : On average, restaurants with one Michelin startend to be rated higher by the public than similarnon-starred restaurants.

Methodology : In order to check this hypothesis, we formalisewhat we mean by “similar restaurants”, and we de-fine an algorithm to look at the difference in theratings. We define two restaurants ri and rj to besimilar if and only if they satisfy the following listof desiderata:

1 ri and rj are in the same city.2 ri and rj have the same price category.3 ri and rj have cuisine similarity

sij =|Ci∩Cj ||Ci∪Cj | ≥ ρ, where ρ ∈ [0, 1] and Ck are

the cuisines associated to rk, k ∈ {i, j}.



Hypothesis 3

Hypothesis 3

Algorithm 1 Restaurant Similarity

1: Fix the value of ρ2: Initialise A which will contain the average difference in the ratings3: Store the information about the Michelin star restaurants into R4: for each one-Michelin-star restaurant r in R do5: Store the rating of restaurant r into m6: Find all similar restaurants (as defined above) that have zero

Michelin stars and store their rating into M7: Compute the mean of M and store it into m8: Store m− m into a9: Append a to A

10: end for11: Take the mean of A and store it into a12: return a


Results

Outline

1 Introduction

2 The Data


4 Results

5 Conclusion


Results

Hypothesis 1

Hypothesis 1

As indicated in table on theright, the Wilcoxon rank sumtest suggests that there is a sta-tistically significant difference inalmost all TripAdvisor ratingsbetween 1 and 2 as well as be-tween 1 and 3 Michelin star(s)restaurants, thus rejecting Hy-pothesis 1.

Table: Wilcoxon rank sum testresults

Rating Groups P-value

Overall1-2 0.003 **1-3 0.007 **2-3 0.205

Value1-2 0.8111-3 0.6392-3 0.563

Service1-2 0.002 **1-3 0.001 **2-3 0.074 **

Food1-2 0.005 **1-3 0.05 *2-3 0.57

Atmosphere1-2 0.003 **1-3 0.007 *2-3 0.2

Red values indicate a negative difference.Significances codes: 0 *** 0.001 ** 0.01 *


Results

Hypothesis 1

This proves that the reputation of Michelin stared restaurantsis not homogeneous across the number of stars according to Tri-pAdvisor users.


Results

Hypothesis 2

Hypothesis 2

MLogit : The marginal effect on the difference between theoverall rating and the number of Michelin stars is

Negative w.r.t.:

Average restaurant expenditureNumber of helpful votes for Michelin relatedreviews

Positive w.r.t.:

Number of visits to one, two or threeMichelin stars restaurantsUser levelMean review sentiment


Results

Hypothesis 2

Hypothesis 2

MLP : Confusion matrix

Precision Recall F1-score Support

Negative 0.72 0.03 0.07 846Positive 0.73 0.64 0.68 16510Very positive 0.65 0.78 0.71 14085

Avg / total 0.70 0.69 0.68 31441

The model which generated this classification, waschosen in accordance to the grid search results whichindicated that the best performing model in termsof negative logarithmic loss was parametrised as fol-lows:

Four hidden fully connected layers eachhaving 100 neuronsA learning rate of 0.001A regularisation strength of 0.01


Results

Hypothesis 2

Hypothesis 2

Overall, we have shown how certain background information aboutonline reviewers can help explaining as well as predicting the pos-sible differences between eWOM, in terms of overall rating givento a restaurant, and expert opinion in terms of Michelin stars.

Hence, we can accept Hypothesis 2.


Results

Hypothesis 3

Hypothesis 3

By running Algorithm 1 withdifferent values of ρ , which con-trols the cut-off point for con-sidering two restaurants similarcuisine-wise we can see how themean difference changes accord-ingly. From Figure 3 it can beseen how the difference is alwayspositive regardless of the value ofρ (it fluctuates between approx-imately 0.12, and 0.195), thusvalidating Hypothesis 3.


Conclusion

Outline

1 Introduction

2 The Data


4 Results

5 Conclusion


Conclusion

Conclusion

We highlight that one Michelin star restaurants are “cursed” inthe sense that they have significantly lower overall, service, foodand atmosphere ratings when compared to two and three Miche-lin star restaurants.


Conclusion

Conclusion

We find that the marginal effect of the average restaurant expen-diture and number of helpful votes for Michelin related reviewson the difference between the overall rating and the number ofMichelin stars is negative, whereas the effect of the number of vis-its to one, two or three Michelin stars restaurants, user level, andmean review sentiment is positive. Further, this study presentsan MLP classifier which is able to classify with 70% precisionwhether the rating difference is negative, positive, or very posi-tive.


Conclusion

Conclusion

We show that, on average, one star Michelin restaurants have ahigher mean rating when compared to similar (same city, pricecategory and cuisine tags) non-starred restaurants.


Conclusion

The End

The End,

Thank You.

Date post:	27-Mar-2018
Category:	Documents
Upload:	doankhanh
View:	223 times
Download:	1 times

The Michelin Curse: Expert Judgment Versus Public … Michelin Curse: Expert Judgment Versus Public...

Documents