Post on 09-May-2015
description
transcript
Copyright 2010 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute
www.deri.ie
Building a Standpoints Web to Support Decision-Making in
Wikipedia
Jodi Schneider
Doctoral Colloquium at CSCW 20122012-02-12Seattle, Washington
Digital Enterprise Research Institute
www.deri.ie
What I’m looking for
Scoping & focus Detailed mentoring on CSCW/HCC methodologies Interviewing Qualitative Research Statistics
Suggestions for evaluating my work
2
Digital Enterprise Research Institute
www.deri.ie
3
Digital Enterprise Research Institute
www.deri.ie
Should we delete this article?
4
Digital Enterprise Research Institute
www.deri.ie
Improving deletion discussions
Main problems: Newcomers who don’t know how to argue Overwhelm of long discussions Discussions that happen over and over again
Deletion as quality control Large number of discussions - ~500/week
5
Digital Enterprise Research Institute
www.deri.ie
Deletion argument
[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.
>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
6
Digital Enterprise Research Institute
www.deri.ie
Goals
Newcomers who don’t know how to argue Characterize the “good” and “bad” arguments Develop argument templates Provide guidance and support for new users in properly structuring arguments according to Wikipedia’s rhetorical standards
Overwhelm of long discussions Develop a claims/argument explorer
Discussions that happen over and over again Prototype an argument bot Populate argument maps with mixed-initiative claims extraction
7
Digital Enterprise Research Institute
www.deri.ie
Overview
Corpus: All Wikipedia deletion discussions from January 29, 2011
Perspectives/approaches: Argumentation CSCW/HCC Text analytics Ontologies/Social Semantic Web
8
Digital Enterprise Research Institute
www.deri.ie
Current work
Analysis of the corpus Argument schemes (e.g. expert opinion) Factors (e.g. notability, uniqueness) Newcomer’s arguments
Interviews Administrators Experienced users
Argument exploration Text mining cue words (‘however’, ‘therefore’,…)
Architecture Ontology development “Standpoints Web”
9
Digital Enterprise Research Institute
www.deri.ie
Standpoint
[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.
Proposition: does not merit an article
Justification: hasn’t played since 2008, bad record
10
Digital Enterprise Research Institute
www.deri.ie
Opposing standpoint
>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
Proposition: keep the article
Justification: meets our article criteria
11
Digital Enterprise Research Institute
www.deri.ie
Possible applications
Visualize decision-making Highlight controversies Query opinions and arguments Discuss arguments interactively with a bot Calculate the “best” options Analyze, extract, and represent disagreement
13
Digital Enterprise Research Institute
www.deri.ie
Thanks!
jodi.schneider@deri.orghttp://jodischneider.com/jodi.html@jschneider
Digital Enterprise Research Institute
www.deri.ie
Acknowledgments
Thanks to our collaborators! Katie Atkinson, Trevor Bench-Capon, Adam Wyner (Liverpool)
DERI Social Software Unit Rhetorical Structure, W3C Health Care and Life Sciences
Funding Science Foundation Ireland Grant No. SFI/08/CE/I1380 (Líon-2)
Short-term scientific mission (STSM 1868) from the COST Action ICO801 on Agreement Technologies
15
Digital Enterprise Research Institute
www.deri.ie
Digital Enterprise Research Institute
www.deri.ie
“ELIZA for arguments”
17
Snaith, Lawrence, & Reed, “Mixed initiative argument in public deliberation,” ODET 2010
Digital Enterprise Research Institute
www.deri.ie
Highlight Controversies
Ennals, R., Trushkowsky, B., & Agosta, J. M. (2010). Highlighting Disputed Claims on the Web. In WICOW at WWW 2010.
Digital Enterprise Research Institute
www.deri.ie
Transform Debates into Argument Frameworks
(1) Households should pay tax for their garbage.
(4) (1) Paying tax for garbage increases recycling, so households should pay.
(3) (1)Recycling more is good, so people should pay tax for their garbage.
19
Arrow: premise
Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic. EGOV 2010
Digital Enterprise Research Institute
www.deri.ie
Calculate best options (non-contradictory opinions)
20
Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic.
EGOV 2010
Digital Enterprise Research Institute
www.deri.ie
Claims Extraction
Cue words (“Hence Jaffa Cakes are cakes.”) [Marcu]
Rhetorical Structure Theory
Claims Extraction
Digital Enterprise Research Institute
www.deri.ie
Important Relationships
Attacks Supports
23
Digital Enterprise Research Institute
www.deri.ie
Case Study
Understand: interviews, observation, and content analysis
Intervene: Implement & test the Standpoints Web architecture on Wikipedia deletion discussions
Evaluation:Community feedback
Ontology fitness-for-purpose Precision & recall?
25
Digital Enterprise Research Institute
www.deri.ie
Problem Possible Uses of a Knowledge Representation
Concrete Examples Some Current Directions
26
Digital Enterprise Research Institute
www.deri.ie
The Problem
The Web is full of opinions & commentary. A lot of it disagrees. How do we learn from other people, when they disagree?
27
Digital Enterprise Research Institute
www.deri.ie
My Approach
Identify peoples’ views Collect the explanations people give Create a hypertext web of these views & explanations
28
Digital Enterprise Research Institute
www.deri.ie
Two Persuasive Messages
29
Digital Enterprise Research Institute
www.deri.ie
Why: Walton’s dialogue types
30
Digital Enterprise Research Institute
www.deri.ie
Knowledge-based Claims Vary
Use of statistics & impersonal information
31
Digital Enterprise Research Institute
www.deri.ie
Versus personal appeals…
Where opinions and personal values are explicit
32
Digital Enterprise Research Institute
www.deri.ie
Why: Knowledge, Emotion, Values as a Proxy
33
Digital Enterprise Research Institute
www.deri.ie
Purpose-related keywords
Knowledge statistics
Values truth secret
Rhetoric you can thank
Judgment/Opinion eradicate tough rejecting
34
Digital Enterprise Research Institute
www.deri.ie
Purpose matters
Knowledge-oriented discussions are straightforward to reuse
Opinion-oriented discussion types may require caveating or balancing emotion makes a discussion more interesting can also indicate the potential for bias.
35
Digital Enterprise Research Institute
www.deri.ie
Twitter: Standpoint
Difference between cakes and biscuits? When stale, cakes go hard, biscuits go soft. Hence Jaffa Cakes are cakes. (Was official EU ruling).
View: Jaffa Cakes are cakesJustification: official EU ruling; go hard when stale
36
Digital Enterprise Research Institute
www.deri.ie
Visualize
bCisive