Analysing a Complex Agent-Based Model Using Data-Mining Techniques

transcript

Claire Little, Bruce EdmondsCentre for Policy Modelling

Manchester Metropolitan University

Ed Fieldhouse, Laurence Lessard-PhillipsInstitute for Social ChangeUniversity of Manchester

An “Inconvenient Truth”

• That the universe is not arranged for our benefit (as researchers studying it)

• in other words, that assumptions such as the following are likely to be wrong:– Our planet is the centre of the universe– Risky events follow a normal distribution– Humans act as if they followed a simple algorithm– Society can be understood using simple, universal prinicples– etc. etc.

• In particular, the assumption that there will always (if you look hard enough) be models that are:– (a) simple enough for us to understand and– (b) adequate to what we want to model

• …is wrongComplexity and Context-Dependency, Bruce Edmonds, ECCS, Lisbon, Sept 2010. slide-2

The Alternative

• Thus consider the alternative, more realistic, situation where one is facing some phenomena where any model that is adequate (w.r.t. our goals) will be too complex for us to completely understand

• Instead of indulging in wishful thinking this paper looks at ways forward under complexity

• In other words, if we have a simulation model that is too complex to completely understand, how can we obtain some useful understanding of its properties…

• …and hence use it to leverage some understanding/control over the target phenomena

Analysing a Complex Agent-Based Model Using Data-Mining Techniques. Little & Edmonds, SSC 2014, Barcelona, 3

The Broad Idea

1. Make relatively complex simulations based on available evidence (a “KIDS” approach)

2. Analyse this simulation in a number of ways, including…

3. Data mining output data in a more holistic manner over a broad “space” of settings

4. Look for patterns in the data that suggest (maybe context-dependent) hypotheses

5. These are candidates for simpler (but maybe partial) models of the simulation

The Model

• To explore the complex mix of factors, structures and processes that affect whether people vote

• An agent-based model, with demographics and dynamic social networks

• Was formulated using a mixture of qualitative, survey data and others’ expert opinion/results

Discuss-politics-with person-23 blue expert=false neighbour-network year=10 month=3

Lots-family-discussions year=10 month=2

Memory

Level-of-Political-Interest

Ethnicity

ClassActivities

An Agent’s Memory of Events

Overall Structure of Model

Underlying data about population composition

Demographics of people in households

Social network formation and maintenance (homophily)

Influence via social networks• Political discussions

Voting Behaviour

Technique

• Instead of initiating ‘thin’ analyses of the simulation behaviour (e.g. 1 or 2D parameter sweeps/correlation models against a few key output measures)

• To sample a multi-dimensional space of settings and cluster on a multi-dimensional space of output indicators (in this case 9 parameter x 13 output measures)

• Look at the patterns between clusters for indications as to hypotheses of behaviour

• Then test these with targeted simulation experimentsAnalysing a Complex Agent-Based Model Using Data-Mining Techniques. Little & Edmonds, SSC 2014, Barcelona, 7

More Holistic but Less Detailed

Multi-Dimensional Space of Parameter

Settings

1. Many Runs, Randomly Sampling Parameter Values

3. Look for patterns that you might then check in a more

systematic manner

2. Analyse data set of result measures using data mining

Parameter ranges

3862 independent runs with parameters sampled from the following, uniform distributions:

• density: [0.65, 0.95]

• drop-activity-prob: [0.05, 0.15]

• drop-friend-prob: [0, 0.01]

• emmigration-rate: [0 ,0.03]

• immigration-rate: [0, 0.02]

• int-immigration-rate: [0, 0.02]

• majority-prop: [0.55, 1]

• prob-move-near: [0, 1]

• prob-partner: [0.01, 0.03]Analysing a Complex Agent-Based Model Using Data-Mining Techniques. Little & Edmonds, SSC 2014, Barcelona, 9

For each of these runs…

• Measure many different indicators of the outputs (say at the end of the simulation) including:– Pop.size – population size– Av.age – average age– Av.adfriends – average number of friends (adults only)– Prop.maj – proportion of the majority population– Prop.adult – proportion that is adult– Prop.1stgen – proportion that are 1st generation immigrant– av.clust – average proportion of friends who are friends– av.sim.hh – average similarity within households– av.sim.fr – average similarity between friends– ncvs.ac – number of conversations over activity links– ncvs.sc – number of conversations over “school” links– Prop. Adults with highest level of political interest

Dendrogram of hierarchical clustering of simulations

A heatmap of the hierarchical clustering

The within group sum of squares against the number of clusters for 10 randomly initialised runs using k-means

Clustergram of PCA-weighted mean of k-mean clusters vs. number of clusters

Centroid plot against the first two discriminant functions showing the 3 clusters

Details of the centroids of the 3 k-means clusters

the 3 clusters against the parameters: emigration rate, immigration rate, internal immigration rate

Multi-Dimensional Scatter Graphs

Pop Size Av. Age

Average proportion of similar friends against time for different immigration rates

Average link density against time for different initial majority proportions

Conclusions

• The particular results and insights in this model are not as important as the overall approach which…

• …tries to get a more complex and holistic idea of the properties of a complex model

• …which then might suggest simple hypotheses/models

• …and thus “stage” abstraction a bit more gradually and carefully, being more aware of what is being abstracted away

Postscript: Emerging Principles

• That evidence should not be ignored without a very, VERY good reason

• That abstraction should be staged in gradual steps rather than “heroic” leaps

• Be clear and explicit about your goals• Separate exploratory from analytic stages• Recognise that it is easy to fool ourselves

and impose (wrong or limited) assumptions• Utilise any and all techniques that are

applicable, but recognising their limitationsAnalysing a Complex Agent-Based Model Using Data-Mining Techniques. Little & Edmonds, SSC 2014, Barcelona, 22

The End

Claire Little: http://

Bruce Edmonds: http://bruce.edmonds.name

Centre for Policy Modelling: http://cfpm.org

Ed Fieldhouse: http://

Laurence Lessard-Phillips: http://Institute for Social Change: http://

The SCID Project: http://www.scid-project.org

These slides will be at: http://slideshare.com/BruceEdmonds

The simulation will ‘soon’ be at: http://openabm.org as “The Voter Model”

Analysing a Complex Agent-Based Model Using Data-Mining Techniques

Education