Towards a classification framework for social machines

Submission at SOCM2013@WWW2013

Elena Simperl 26 April 2013

Motivation and objectives • Future ICT systems as sophisticated assemblies of data-intensive, complex automation and

deep community involvement

• Defining social machines and their characteristic properties as necessary step towards a principled understanding of the science and engineering of such systems

• Objectives of this work

– Identify and define the constructs to describe, study, and compare social machines

– Achieve a shared understanding of basic notions and terminology through involvement from the broader community

• Useful tool for both researchers in social and computer sciences and for developers and operators of existing and future social machines

2

General considerations • Machine: ‘(1) an assemblage of

parts that transmit forces, motion, and energy one to another in a predetermined manner; (2) an instrument (as a lever) designed to transmit or modify the application of power, force, or motion’ [Merriam-Webster]

• In relation to living beings: ‘one that resembles a machine (as in being methodical, tireless, or consistently productive)’ [Merriam-Webster]

• Social machine

1. co-existence of and interaction among algorithmic and social components;

2. problem/task specification changes as the system evolves;

3. operation of the system is governed by a different set of rules;

4. different performance models and approaches to measure them;

3 [Courtesy of Dave de Roure]

The polyarchical relationship of social machines

• Platforms/technologies vs social machines created for specific purposes. E.g., MediaWiki vs Wikipedia

• Broader vs narrower-scoped social machines. E.g., Twitter vs Obama’12

• Ecosystem of social machines. E.g., results from GalaxyZoo taken up in Wikipedia articles

4

Social machines and related areas • Computer science:

CSCW, social computing, human computation

• Organizational management/social sciences: wisdom of the crowds, collective intelligence, open innovation, crowdsourcing

5

Social machines and related areas (2) • Who defines the task/purpose of the system

– The system designer vs community

• What kind of tasks do humans undertake

– Creative vs computationally expensive

• Who is supporting whom

– Humans supporting algorithmic processes or machines supporting human tasks

6

Methodology • Repertory grid elicitation to derive an initial set of elements

(instances of social machines) and constructs (characteristics of social machines) 10 grids, 56 elements, 117 constructs

• Consolidation and clustering of constructs 31 constructs, five clusters

– General description – Purpose and tasks – Participants and roles – Motivation and incentives – Technology

7

Purpose and forms of contribution • Contributions towards public vs private good

• Implicit vs explicit contributions

• Degree to which contributors decide what they can work on

• Degree to which contributors can change the nature/purpose/development of the social machine

• How is the final result created/aggregation

8

Participation and interaction • Who can contribute and what: roles, requester/worker,

game models, skills and learning curve

• Workflow management: task/resource assignment (scarcity, requester-contributions cardinality), parallelization, synchronization, aggregation

– Machine replacing/assisting humans vs humans replacing machines

• Dynamics of participation model

9

Quality and performance • Which contributions are validated

• Is there a ground truth and where does it come from: no one, community, dedicated group, machine owner

• How is quality assessment performed: manually, agreement/voting between participants, computed automatically

• Are criteria and quality control methods explicit/transparent

• Can contributors change the criteria or earn the right to perform evaluations

10

Motivation and incentives • Altruism

• Reciprocity

• Community

• Reputation

• Autonomy

• Entertainment/Fun

• Intellectual challenge

• Learning

• Competition

• Payment/Rewards

• Depend on

– Nature of the good produced

– Goal

– Nature of the contributions

– Existing social structure

11

Technology and engineering • Requirements specification and evolution

• Security, trust

• Decentralization

• Data ownership and access

• Profile building

• Social networks

• Analytics on top of social network and actual data

12

[Courtesy of Dave Robertson]

Next steps • Consolidate and use the classification

• Evaluation

– Task-independent using criteria from knowledge engineering (completeness, correctness, readability, redundancy etc)

– Task-dependent: Can the framework be used to describe existing social machines?

13

Theory and practice of social machines May 13, 2013

14

Constructs: purpose of the system and contributions

• Purpose of the system, types of contributions, degree to which these change

15

Constructs: people, roles, motivation • Types of audience, autonomy and anonymity, roles and role

hierarchies

• Intrinsic vs. extrinsic motivation, rewards

16