Including the Human in Humanitarian Big Data
Sara-Jayne Farmer@bodaceacat
The UN’s Development Reversal Problem
2
Where we’d like to be
3
How bad is it? 2008 – 2011 information lags
4
2008: Not knowing what, who, where
5
2011: New Data Sources
6
But new data only gets us so far…
7
Bots
Humans
Humans don’t have all the info, machines don’t have all the smarts
Hunchworks: hypothesis management
» System = people + processes + tech
» Break down the siloes: Engage and connect the global network of UN and international development field staff with analysts and decision makers
» Use the machines better: use agents to detect weak signals
» Use the people better: Encourage ‘hunches’ (hypotheses) about anomalous or interesting data and observations
» Encourage cooperation: Rapidly collect information about emerging situations
» Combine strengths: mix human insight with data analysis results
8
Hunch Mechanism
» Detect an interesting “weak signal”
» Hypothesize (make a hunch) about the nature of the signal
» Share the hunch with your social graph
» Attract related signals and evidence
» Verify hunch by engaging communities of practice
» Act
» Use confirmed hunch to refine trust within the system
9
Issues: Community, Complementary skills
10
Issues: Encouraging use, encouraging Trust
11
Issues: Encouraging Action
12
“is a hunch true enough to take action?”
Issues: Security, localisation, bandwidth
13
Issues: fun problems we’re working on
14
» Hunch set management• Clustering: hunches are related• Merging: hunches are the same thing• Splitting: hunch has changed significantly
» Mixing human and bot users• Autonomy: how to share information, decisions, responsibility• Managing culture differences between humans and bots
» Managing credibility• Rating hunches, users, data• Overall and friends’ ratings
» Hunch network as a reasoning system • vs “a hunch is the discussion surrounding it”