Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 1

Using Data Integration Modelsfor Understanding Complex Social Systems

Bruce EdmondsCentre for Policy Modelling

Manchester Metropolitan University Business School

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 2

Talk Outline

1. Some of the thinking behind this work

2. About Agent-Based Simulation

3. Data Integration Models

4. An example from the SCID project: Voter Turnout

5. Concluding Discussion

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 3

Underlying Principles

• Being “scientific” means not ignoring evidence (at least without a very VERY good reason)!

• This means not ignoring qualitative evidence and not ignoring quantitative evidence

• Also that if theory and evidence clash then (almost always) one should go with the evidence

• Using formal (i.e. ultimately precise) models means that the process of knowledge formation can be far more social

• These models are open to a process of critique, communication without error, and collaboration over a long period of time and many researcher

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 4

However…

…social phenomena are so complex that• There is no reason to suppose we happen to have

brains with the capability of understanding it in a scientific manner

• It is inevitable that we will have to make many compromises in obtaining any useful knowledge about it

• So “heroic leaps” to simple and supposedly general theories will not work

• Rather we will have to settle for complex and situation-specific formulations

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 5

Agent-Based Simulation

• Is a computer program• Much like a multi-character game, where each

social actor is represented by a different “agent”• These agents can each have very different

behaviours and characteristics• Social phenomena (such as social networks) can

emerge out of the decisions and interaction of these individual agents (upwards “emergence”)

• But, at the same time, the behaviour of individuals can be constrained by “downwards” acting rules and social norms from society and peers

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 6

Why Computer Simulations?

• Agent-based simulations are more expressive than analytic mathematics ones, that is, they do not require strong assumptions to work (unlike economics)

• They are largely theory-free, that is they can implement a wide range of different kinds of accounts, hence allowing a more naturalistic style of representation

• They can be very detailed, allowing representation and exploration of some of the meso-level complex mess that much social phenomena consists of (unlike system dynamic models)

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 7 Social influence and the domestic demand for water, Aberdeen 2002, http://cfpm.org/~bruce slide-7

System Dynamics or Statistical modelling

Real World Equation-based Model

Actual Outcomes

AggregatedActual Outcomes

AggregatedModel Outcomes

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 8

Individual- or Agent-based simulation

Real World Individual-based Model

Actual Outcomes Model Outcomes

AggregatedActual Outcomes

AggregatedModel Outcomes

Agent-

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 9

What happens in ABS

• Entities in simulation are decided on• Behavioural Rules for each agent specified (e.g. sets of

rules like: if this has happened then do this)• Repeatedly evaluated in parallel to see what happens• Outcomes are inspected, graphed, pictured, measured

and interpreted in different ways

Simulation

Representations of OutcomesSpecification (incl. rules)

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 10

E.G: Schelling’s Segregation Model

Schelling, Thomas C. 1971. Dynamic Models of Segregation. Journal of Mathematical Sociology 1:143-186.

Rule: each iteration, each dot looks at its 8 neighbours and if less than 30% are the same colour as itself, it moves to a random empty square

Segregation can result from wanting only a few neighbours of a like colour

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 11

Micro-Macro Relationships

Micro/ Individual data Qualitative, behavioural, social psychological data

Theory, narrative accounts

Social, economic surveys; Census Macro/ Social data

Simulation

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 12

The Micro-Macro Link

• How do the tendencies, abilities and observed behaviour of individuals…

• …relate to the measured aggregate properties of society?

• Social Embedding etc. implies this link is complex• Averaging assumptions (a general tendency + random

noise) do not capture non-linear interaction• This is often two-way, with society constraining and

framing individual action as well as individual constituting society in an emergent fashion

• Somewhat-persistent, complicated meso-level structures mediate these effects – these might be key to understanding this

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 13

In Vitro vs In Vivo Analogy

• In biology there is a well established distinction between what happens in the test tube (in vitro) and what happens in the cell (in vivo)

• In vitro is an artificially constrained situation where some of the complex interactions can be worked out…

• ..but that does not mean that what happens in vitro will occur in vivo, since processes not present in vitro can overwhelm or simply change those worked out in vivo

• One can (weakly) detect clues to what factors might be influencing others in vivo but the processes are too complex to be distinguished without in vitro experiments

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 14

Data Integration Models

• Are a particular style of agent-based simulation• Intended more as a computational description of a

particular case than a (generalistic) theory• Its aim is to represent as much of the relevant

evidence as possible in one coherent and dynamic simulation

• Provides a precise target for abstraction (which are then checkable against it)

• Stages abstraction from data to theory• Separates representation and abstraction• Preserves chains of reference

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 15

Aims and Objectives of DIM

• To develop a simulation that integrates as much as possible of the relevant available evidence, both qualitative and statistical (a Data-Integration Model – a DIM)

• Regardless of how complex this makes it• A description of a specified kind of situation (not a

general theory) that represents the evidence in a single, consistent and dynamic simulation

• This simulation is then a fixed and formal target for later analysis and abstraction

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 16

So what kind of explanation does this approach facilitate?

• At the micro-level causation is explicitly specified: what happens under what circumstances with what probability/mechanism?

• At the macro-level, different factors/measures/variables/outcomes can be correlated with each other just as with in vivo macro studies, however you have the possibility of doing controlled experiments!

• However the simulation itself provides an inspectable instantiation of the micro-level events and interaction– Social processes are still deeply entangled and not necessarily

separable– Local situation is important in understanding meso-level causal

explanations

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 17

The “SCID” Project

The Social Complexity of Immigration and Diversity is a 5-year project with the Institute for Social Change and the Department of Theoretical Physics at University of Manchester. It is funded under the “Complexity Science for the Real World” initiative of the EPSRC to the tune of £2.7 million and will last until August 2015.

The idea of the project is to apply the techniques and tools of complexity science to real world issues, in this case of immigration and diversity. The project will focus on: (1) why people bother to go out and vote and how social influence within/across different communities affects this (2) how people use social networks to find employment, e.g. how the impoverished networks of immigrants may limit this and (3) inter-community trust.

Project Website:http://scid-project.org/

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 18

the SCID Modelling Approach

Data-Integration Simulation Model

Micro-Evidence Macro-Data

Abstract Simulation Model 1

Abstract Simulation Model 2

SNA Model Analytic Model

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 19

Overall Structure of Model

Underlying data about population composition

Demographics of people in households

Social network formation and maintenance (homophily)

Influence via social networks• Political discussions

Voting Behaviour

Inpu

t

Out

put

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 20

Discuss-politics-with person-23 blue expert=false neighbour-network year=10 month=3

Lots-family-discussions year=10 month=2

Etc.

Memory

Level-of-Political-Interest

Age

Ethnicity

ClassActivities

A H

ou

se

ho

ld

An Agent’s Memory of Events

Etc.

Changing personal networks over which

social influence occurs

Composed of households of individuals initialised from

detailed survey data

Each agent has a rich variety of individual (heterogeneous)

characteristics

Including a (fallible) memory of events and influences

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 21

Example Output: why do people vote (if they do)

Intervention: voter mobilisation

Effect: on civic duty norms Effect: on habit-

based behaviour

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 22

Simulated Social Network at 1950

Established immigrants: Irish, WWII Polish etc.

Majority: longstanding ethnicities

Newer immigrants

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 23

Simulated Social Network at 2010

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 24

Psuedo-Narrative Output

Following a single, randomly chosen agent…

4: (person 578)(aged 5) started at (school 1)

17: (person 578)(aged 18) stops going to (school 1)

21: (person 578)(aged 22) moved from (patch 11 3) to (patch 12 2) due to moving to an empty home

21: (person 578)(aged 22) partners with (person 326) at (patch 12 2)

24: (person 578)(aged 25) started at (workplace 8)

24: (person 578)(aged 25) voted for the blue party

29: (person 578)(aged 30) voted for the blue party

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 25

Possibilistic vs Probibilistic

• The idea is to map out some of the possible social processes that may happen

• Including ones one would not have thought of or ones that have already happened

• The global coupling of context-dependent behaviours in society make projecting probabilities problematic

• Increases understanding of why processes (such as the spread of a new racket) might happen and the conditions that foster them

• Complementary to statistical models and natural language formulation and discourse

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 26

Using Qualitative Behaviour to Inform the Agent Specification

• Narrative data (from semi-structured interviews, observations etc.) can be used to inform the behavioural rules of agents within these simulations

• This can be done in an informal or semi-formal manner (e.g. using techniques extended from GT)

• This can provide a broader “menu” of possible behaviours and strategies that are used and thus import some of the “messiness” of social reality instead of overly neat formulations (e.g. economic)

• Meso-level outcomes can be fed back using participatory techniques to aid validation

• Macro-level measures can also be extracted and compared to known quantitative data

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 27

Context-Dependency

• In the simulation (as in our social life) decisions, adaption, communication, learning all take place within a local context

• Both “upwards” (emergent) and “downwards” (social control) forces operate within local contexts allowing social embeddedness

• Abstraction to aggregates (e.g. averages) only takes place post-hoc (just as in social statistics)

• Thus ABS allows the formal representation of context-dependent behaviour, albeit within a more specific “descriptive” simulation, that can be itself hard to understand

• Thus opening the way to the study of context itself!

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 28

Conclusion

• Complex simulations are a different way of representing social phenomena than mathematics, data and natural language

• It by-passes the need for overly simplistic assumptions and allows for a more “naturalistic” manner of respresentation, e.g.: heterogenity, and dynamic/emergent social structure/phenomena

• It allows micro-behaviour to be context-dependent – not requiring this to be dealt with as “random”

• The micro-level relates to qualitative evidence, the meso to social networks and the macro to quantitative statistics in a well-founded manner

Using Data Integration Models for understanding complex social systems, Bruce Edmonds, MMUBS, 5th Feb 201. slide 29

Thanks!

Centre for Policy Modelling

http://cfpm.org

I will make these slides available at: http://www.slideshare.net/BruceEdmonds