Peer Ratings in Massive Online Social Networks

Peer Ratings in Massive Online Social Networks

Dmitry Zinoviev

Suffolk University

Boston

Sunbelt-2012

SUNBELT-2012 / SUFFOLK UNIVERSITY 2

To Like or not to Like?

● Like/dislike ratings provide instant peer feedback in massive online social networks (MOSNs)

● Objects subject to ratings: posts; photographs; comments● Questions:

➢ How many stops should rating scales have?

➢ How do MOSN users perceive the ratings?

➢ How do MOSN users react to the ratings?


Rating Types

● Single-valued (Facebook ”Like”, Google+ ”+1”); do not allow negative attitudes

● Binary a.k.a. ”Thumbs Up/Down” (Pandora, Yahoo Answers); do not allow fuzzy answers

● Multivalued (Yahoo, Amazon); may be hard to use if there are many stops; these ratings are of particular interest to us


“Odnoklassniki.Ru”—Our Testing Range

● Largest Russian-language social network (100 mln users, 31 mln daily visits)

● Most users have more than one personal picture and reasonably credible demographics (age/gender)

● 6-way ratings (“1” through “5+”*) applicable to personal pictures● Anyone can leave comments to any personal picture of to the profile

in general● The system records and displays all profile visits (and visitors)

*The grade of “5+” requires a symbolic payment from the grader


A Profile, as Seen by a VisitorMy PageMy Page

Personal PhotographsPersonal Photographs

DemographicsDemographics

FriendsFriends

Profile PictureProfile Picture


Picture Rating Interface

PicturePicture

Picture CommentsPicture Comments

DemographicsDemographics

This is a ”5+” pictureThis is a ”5+” picture

Your ratingYour rating

DescriptionDescription


Method

● Select a random avatar Q from the three avatars on the right (30/Female, 40/Male, 30/Male)

● Select a random currently logged user and record the user's age A (≥18) and gender G

● Select a random grade Rin between “1” and “5+” and post it

● Record the user's response:➢ did the user visited the test profile? (V)

➢ did the user post a comment? (C)

➢ what is the response grade Rout

?

● Apologize to the user for a low grade. Posted grades and comments can be easily removed without any traces


Descriptive Statistics

● 10,799 observations● 3,600 observations per avatar● 1,800 observations per grade● Distribution by gender:

➢ 54% female

➢ 46% male

● Mean age 34

Russia in 2009


Empirical Parametrized Mapping

{Rout

, V, C} = f (Rin; Q, A, G )

Free variable

Parameters


Response Details with Trend Lines

Responders, by kind and by age:


Response Statistics

All RespondersOf them:

Visitors Graders

RateM 47.0% 39.7% 19.7% 4.8%

F 42.4% 31.7% 19.5% 5.0%M -0.1% 0.0% -0.4% -0.1%

F 0.5% 0.6% -0.1% 0.1%

Commenters

Age Trend (%/year)

● Visits are the most common responses, followed by grades, followed by comments

● Male subjects visit the experimenter's profile more often● Older female visitors are more active than younger● Younger male graders are more active than older● Grading/commenting rates are gender-agnostic


OutGrade Distribution

● The grades of “5” and “5+” are grouped (“5+” is special because of its associated price)

● Response grades have a sharp bi-modal distribution● Only grades “1” and “5” matter!


OutGrades vs InGrades

● Response grades are sharply distributed around “1” and “5” for any stimulus grade (the red lines show best-fit bi-exponential distributions)

● Only two stops on the scale are necessary!


RG=R+G

● Response Grade = Reciprocity + Generosity➢ Reciprocity: Social norm of in-kind responses to the behavior of

others (a.k.a. “eye for an eye”)

Responding with the same grade➢ Generosity: Habit of giving freely without expecting anything in

return

Responding with a better of worse grade (positive vs negative generosity)

● How popular are these mechanisms?


Reciprocity (1)

● Define reciprocity as the “fraction of reciprocally equal grades out of all grades”:

● Calculate reciprocity for each avatar and for each age-gender group● Calculate linear best-fit estimate● Notation:

➢ Blue lines for male subjects, red lines for female subjects

➢ Solid lines for the 40/M avatar, dashed lines for 30/M, dotted lines for 30/F

Rec=N Ri n=Rout

N


Reciprocity (2)

● Older subjects are less reciprocating

● Male subjects are on average less reciprocating

● Younger avatars generate more reciprocity from the subjects of the same gender


Generosity (1)

● Define generosity as the average value of the difference between the stimulus grade and the response grade:

● It can be positive and negative● Calculate generosity for each avatar and for each age-gender group● Calculate linear best-fit estimate

Gen=∑i

Rout−Ri nN


Generosity (2)

● Older subjects are more generous

● Older avatar generates less generosity

● Subjects are less generous to the avatars of the same gender

● Younger avatars generate less generosity from the subjects of the same gender


Benevolence (1)

● Let negative comments have the value of -1, positive comments—the value of 1, and neutral comments—the value of 0. Define benevolence as the average value all comments

● Calculate benevolence for each avatar and for each age-gender group

● Calculate linear best-fit estimate


Benevolence (2)

● Older subjects leave better comments

● Female subjects leave better comments

● Younger avatars get worse comments from the subjects of the same gender


Grading/Commenting Overview (1)

Generosity G

Reciprocity R

Benevolence B

● Most behaviors are avatar-specific; however, the dependency on age is universal


Grading/Commenting Overview (2)

● Generosity and benevolence grow with age● Generosity and benevolence are weaker for same-gender

avatar-subject pairs● Reciprocity slowly declines with age● Reciprocity is stronger for same-gender avatar-subject pairs● All three values can be approximated using quadratic functions


Latent Hostility

● Age 18–22: subjects are all negative

● Age 22–36: subjects combine positive generosity and negative benevolence: they post higher grades but leave negative comments

➢ Hypothesis: Higher response grades make subjects feel good, but comments reveal true feelings

● Age 36–80 subjects are all positive

18–22

36–80

22–36


THANK YOU!

Date post:	08-Jul-2015
Category:	Technology
Upload:	dmitry-zinoviev
View:	86 times
Download:	0 times

Peer Ratings in Massive Online Social Networks

Technology