Avatars Versus Agents: A Meta-Analysis Quantifying the Effect of Agency on Social Influence
Jesse Fox, Sun Joo (Grace) Ahn, Joris H. Janssen, Leo Yeykelis, Kathryn Y. Segovia & Jeremy N. Bailenson
Presented by: Pantea Habibi
Outline
◉ Overview ◉ Research Design◉ Meta-Analysis◉ Results ◉ Implications◉ Critique & Discussion
2
Overview 1
3
Overview
◉ Virtual representations of people in computer-mediated interactions ○ Avatars ○ Agents○ Hybrids
◉ Ability to behave in a human-like manner◉ unique capabilities to influence users◉ modify users’ attitudes and behaviors
4
Overview
◉ Social influence ■ social norms ■ conformity ■ compliance ■ Identification
5
◉ Social presence◉ processes
○ Identification○ social categorization
◉ perceive the representation
Overview
◉ Social influence ■ social norms ■ conformity ■ compliance ■ Identification
6
◉ social presence◉ processes
○ Identification○ social categorization.
◉ perceive the representation
I have a feeling that there is another dimension to this in that whether it matters would depend on the specificity of the task. For some tasks, it may matter and some others it might not. For some types of tasks, the exact identity might matter and for others, a less specific variation like gender might matter.[Harish]
Research Design2
7
Research purpose
◉ Virtual representations○ education, health, marketing, and other persuasive
contexts○ important to identify the differences
◉ Determine whether perceived agency of a virtual representation affected social influence
8
Variables of Interest
9
◉ Perceived Control◉ To enrich the analyses and application of the findings, relevant
moderators included:○ Level of Immersion
■ Immersive/Desktop○ Type of Measure
■ Objective(behavioral)/Subjective(self-reported) DV○ Task Type
■ Competitive/Cooperative/Neutral○ Actual Control○ Time
Variables of Interest
10
◉ Perceived Control◉ To enrich the analyses and application of the findings, relevant
moderators included:○ Level of Immersion
■ Immersive/Desktop○ Type of Measure
■ Objective(behavioral)/Subjective(self-reported) DV○ Task Type
■ Competitive/Cooperative/Neutral○ Actual Control○ Time
I like how they could dissociate the actual content of the papers and just talk about high level goals of research they wanted to achieve. However, would the context / content affect the meta-analysis? (Comparatively the second paper, explained thoroughly each papers context and how it blended or did not with the other).[Aditi]
Research Question
◉ Will agency effects on social influence vary between objective variables and subjective variables?
◉ Will actual control moderate the effect of perceived control on social influence?
11
Hypothesis
◉ Avatars will yield greater influence than agents.◉ Studies conducted in immersive virtual environments will
demonstrate smaller differences in the effects of agency than studies conducted in desktop environments.
◉ Competitive tasks will show the greatest differences in agency, followed by cooperative tasks, and neutral tasks will show the smallest differences in agency.
12
Meta-Analysis 3
13
Study Selection
14
◉ Bibliographic indices relating○ Virtual reality, communication, psychology, video gaming
◉ Databases: ScienceDirect, Google Scholar, etc.◉ Search terms: Virtual representation, avatars, etc.◉ Once a paper identified:
○ References / reverse search◉ Advertised on relevant listservs and contacted researchers◉ Measured social influence◉ 119 studies
Study Selection
15
◉ Bibliographic indices relating○ Virtual reality, communication, psychology, video gaming
◉ Databases: ScienceDirect, Google Scholar, etc.◉ Search terms: Virtual representation, avatars, etc.◉ Once a paper identified:
○ References / reverse search◉ Advertised on relevant listservs and contacted researchers◉ Measured social influence◉ 119 studies
I like the idea of contacting other researchers to get relevant work. I think that is also a good way to find more relevant papers.[Nina]
Study Criteria
16
◉ DVs - quantitative measures of social influence ○ presence or affect ratings, physiological measures, or
interpersonal distance◉ Have a visual representation◉ Explicitly manipulate agency◉ 36 studies
Final Round of Selection
17
◉ Minimally required statistics:○ means, standard deviations, and sample sizes per
condition○ t values○ F values with degrees of freedom○ r values○ Cohen’s d values
◉ 32 studies
Final Round of Selection
18
◉ Minimally required statistics:○ means, standard deviations, and sample sizes per
condition○ t values○ F values with degrees of freedom○ r values○ Cohen’s d values
◉ 32 studies
I also like how they were able to get sample sizes and needed statistics by contacting authors. But, for the one study that they couldn't get enough information, why didn't they just exclude the study instead of assuming values?[Nina]
I agree with Nina for that one study that assumed the sample sizes but I am not very critical of that assumption as there is a clear selection criterion for the groups (8 and 5 grades), I might be wrong but I arrived at the reasoning based on papers where the American core syllabus for mid and high schoolers was being designed and had a similar assumption.[Sourabh]
“The only case for which we were not able to obtain information was the Eastin and Griffiths (2006) studies, for which we assumed equal sample sizes across conditions.”
Data
19
◉ Average sample size within each study: 71◉ Desktop: 21, immersive VE: 11◉ Tasks:
○ Gaming: 16■ 7 Human, 9 Computer
○ Non-gaming: 16■ 15 Computers, 1 Human
◉ Type of tasks:○ Cooperative: 4○ Competitive: 11○ Cooperative and Competitive: 1
Effect Size Calculations
20
◉ Datasheet○ Studies
■ DVs● Measures (objective/subjective)
○ Statistics + number of participants◉ For each DV, calculated effect sizes
○ r value■ t value
● means, standard deviations, and the number of cases in each condition
Effect Size Calculations
21
◉ Missing r value: ○ Coded two values for r:
■ coded r as zero● avoids inflated results due to publication bias of
significant effects■ maximum non-significant effect size
● based on the sample size◉ Sign of r signifies the direction of the effect
○ Positive: avatar condition yielded stronger responses on the DV than the agent condition
Effect Size Calculations
22
◉ Surveyed studies○ reported more than one DV of social influence
■ conducted an analysis with DV as the unit of analysis
◉ conducted analyses using study as the unit of analysis
Data Sheet Sample
23
Statistical Analyses
24
◉ Calculated r values - weighted by sample size○ weighted r values were transformed to Fisher’s Z values○ After aggregation, the averaged Fisher’s Z values were
transformed back to r values◉ Grand mean and variance of r was calculated
○ first source of variance around the mean r ■ sampling error variance ??■ still unexplained variance left?
● moderator variables◉ Assessed the n-r correlation
Statistical Analyses
25
◉ Calculated r values - weighted by sample size○ weighted r values were transformed to Fisher’s Z values○ After aggregation, the averaged Fisher’s Z values were
transformed back to r values◉ Grand mean and variance of r was calculated
○ first source of variance around the mean r■ sampling error variance■ still unexplained variance left
● moderator variables◉ Assessed the n-r correlation
I did not clearly understand how they calculated the effect sizes they have brief discussion about filling in missing values and the n-r calculations, I feel that the authors not talking about what the dependent variables were in concrete details disrupted this understanding.[Aditi]
Results 4
26
Results
27
◉ Overall effect of agency in which avatars were more influential than agents
◉ Computed Fisher’s Z values from the r values of each DV and submitted all sets of Fisher’s Z values to a one-sample t test
◉ All DVs
Results
28
◉ Average effect size per study
Results
29
◉ Possible factors that influenced the effect sizes◉ Subtracted the sampling error variance from the total variance
○ Resulting variance was greater than zero○ Likely to be moderator variables to explain part of the
variance in the data
Results
30
◉ Effect of year of publication on the effect of agency◉ Regressed year on the four sets of r values
Results
31
◉ Effects of Immersion and DV type
◉ Submitting the Fisher’s Z values of each of the 4 sets to ANOVA
◉ Between-DV factors
Results
32
◉ means and standard errors◉ Pairwise comparisons with Bonferroni corrections
Results
33
◉ Effect of task type on agency◉ Submitted the Fisher’s Z values of all four sets to an ANOVA
with task type as a between-DV factor
Results
34
◉ Pairwise comparisons using Bonferroni corrections◉ r values were higher in both gaming tasks compared to the
non-gaming tasks
Results
35
◉ Actual control○ made any difference on the effect of the perceived control
◉ Fisher’s Z values of all four sets to an ANOVA○ between-DV
Results
36
◉ Investigated the n-r correlation as a measure of publication bias
◉ Significant negative correlations between n and r○ some studies were not uncovered in the analysis
Implications5
37
Implications
◉ Theoretical◉ HCI Research and Design
○ persuade an audience via a virtual representation to convince them they are interacting with a real person rather than an algorithm
○ Adopt multiple methods of measures
38
Implications
◉ Theoretical◉ HCI Research and Design
○ persuade an audience via a virtual representation to convince them they are interacting with a real person rather than an algorithm
○ Adopt multiple methods of measures
39
This might be a good conclusion, might have important implications when designers think about the system.However, the question of how exactly one is supposed to achieve it is as old at the field of computer science itself. Is the goal of the designer here to make sure the software passes the Turing test? Perhaps that was not the idea that the authors were going for, but how else could we put that in the context of avatars vs agents?[Harish]
Critique & Discussion6
40
Critique◉ Strengths
○ agency is an important factor to consider when examining social influence in mediated interactions
○ Multiple Variables
◉ Weaknesses○ Bias of reporting and publishing only significant results○ Relies on the original researchers’ execution and reporting
■ did not report system features○ Few studies - experimental studies - small sample sizes
◉ Possible Improvement○ Role of behavioral realism○ Investigate how an optimal hybrid can be constructed for different contexts○ Variable categorizing 41
Discussion
◉ The paper is easy to read.[Nina]
◉ I think this is a good example of a meta-analysis. It supports what we learned in
class and I had a few of the "aha" moments while reading it.[Nina]
◉ I liked how the paper cited various forms of virtual representation. The
Blascovich study that suggests that user makes a decision without regarding if
the avatar is human or computer controlled was an interesting mention.
Research questions are well defined based on previous studies.[Sourabh]
◉ A very easy to understand example for meta-analysis, selection criteria are well
defined. [Sourabh]
42
Any questions ?
Thanks!
43