Post on 16-Jan-2016
transcript
Information value:the value of evidenceDr David J Marsay, C.Math FIMAA presentation to 19 ISMOR29.08.2002
3
Contents
1 Introduction
2 Examples
3 Theory
4 World-view
5 Implications
6 Conclusions
IntroductionSection 1
5
Introduction
• Information is the life-blood of military C4ISR.
• Any time we prefer one set of information to another we implicitly ‘value’ it.
• We think we could do better:
– lessons identified.
– studies.
• Specifically needed to support UK MOD’s ARP 14 ‘Battlefield picture compilation’.
1
6
Introduction
• We use P(A|B) to denote ‘the probability of A given B’
– P(A) is used to denote the [prior] probability.
• For hypotheses {H} and evidence E:
– Shannon’s ‘entropy’ is calculated from the ‘final’ probabilities, {P(H|E)}.
– Jack Good’s ‘weight of evidence’ is calculated from the likelihoods, {P(E|H)}.
– According to Bayes’ rule, the probabilities can be calculated from the likelihoods and ‘the priors’.
1
ExamplesSection 2
8
Examples
Control: Suppose that a source sends accurate data to a deterministic machine.
• Shannon’s concept does not apply. Nor does the notion of ‘priors’.
• The value of the data can be determined by valuing the function of the machine - no fancy method needed.
• The likelihoods make sense. They are 0 or 1.
2
9
Examples
Command - ‘soft’ aspects:
• For an information artefact (e.g., an INTSUM) to represent the same information implies that all recipients had the same priors. Thus everyone receives everything in the same order.
– Is this realistic?
• Alternatively, one could define some privileged ‘central’ viewpoint for which the information is defined.
– Does this fit doctrine?
– Is it helpful?
2
10
Examples
Command - ‘soft’ aspects:
• The likelihoods {P(E|H)} are a rating of the source of E. They are thus relatively ‘objective’, ‘knowable’ and ‘shareable’.
• Likelihoods relate to current practice (reliability, accuracy).
2
11
Examples
Compilation:
• The work being reported on has looked at the relatively ‘hard’ problem of compilation, particularly ‘Battlefield picture compilation’ under ARP 14.
• Weights of evidence can be used. (See accompanying paper.)
• When is this reliable?
2
Theory
Section 3
13
Theory
Jack Good’s evidence:
• Likelihoods are often straightforward.E.g., P(‘Heads’|‘Fair Coin’) = 0.5 by definition.
• Lab and field testing traditionally establish, in effect, likelihoods.
• Surprise = -log(likelihood).
• Weight of evidence (woe) is surprise, normalised by the prior expected surprise for the same evidence. (So that only ‘relevant detail’ counts.)
3
14
Theory
Evidence is more fundamental than Shannon’s information
• Shannon’s entropy is expected surprise.
• The more useful cross-entropy is likely surprise.
• Woe supports alternative decision methods, such as sequential testing, hypothesis testing.
3
15
Some questionable assumptions
• Shannon assumes that systems of interest are ‘Markov’.
• Shannon noted that ‘state-determined systems’ are ‘Markov’ with probability 1.
• But Smuts (e.g.) noted that evolution drives dynamical systems to adopt synergistic ‘emergent’ structures.
• These had a priori probability 0.
• So for social systems, international relations, military conflict ... we cannot rely on Shannon’s ‘information’.
3
16
Some questionable assumptions
• But can likelihoods be used?
• If we abandon Markov models, how are we to judge if a given algebra of likelihoods is valid?
• We need a ‘world meta-model’ to replace Markov.
3
World-viewSection 4
18
• Allows one to investigate emergence within complex systems.
• Evidence of piece-wise Markov behaviour.
• Developed under the MOD CRP TGs 0,5,10.
SMUTS(synthetic modelling of uncertain temporal systems)
Delayed Double Vipert = 0.0502
BUBs2D5.5
t = 0.2005
4
19
Alternative ideas
• I postulate a model in which systems of interest to the military are Markov in space-time ‘zones’, with more interesting transitions at their boundaries.
• Thus Markov locally, but not globally.
• In essence emergence only happens when an over-adaptation is exploited. (E.g. Ashby, Piaget.)
• Thus, as long as we can learn at least as quickly, we should be able to recognise these situations too.
4
20
Supporting evidence
Applications to, and experiences of:
• warfare
• economics
• international relations.
(My subjective view)
4
21
Reuters data for the Balkans, the 90s4
Evidence of locally Markov behaviour
Balkans April 1989- March 1999KEDS data from www.ukans.edu/~keds
Entropy / Value Aggregated Monthly Phase plot
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0 10 20 30 40 50 60 70 80 90 100
RSS Value
Tra
nsacti
on
al
En
tro
py,
aft
er
Sh
an
no
n
4/89-4/91
4/91-11/91
11/91-10/95
10/95-4/97
4/97-9/97
9/97-1/99
1/99-3/99
ImplicationsSection 5
23
Implications for ‘information’
Technical differences:
• The difference between the expected weight of evidence (woe) and Shannon’s entropy is not a constant.
• Systems of interest tend to have ‘long range’ sources of uncertainty, in addition to the ‘local’ entropy.
• We need to allow for this and ‘expect the unexpected’ to achieve robustness.
5
24
Implications for ‘information’
Some cases where Shannon might not be appropriate
• Poor ‘local’ information.
• The ‘situation’ cannot necessarily be recognised.
• The ‘target’ is adaptable (particularly if adapting against us).
5
25
Implications for ‘information’
Typical symptoms that Shannon is inadequate:
• Mistakes often reflect a need to validate assumptions.
• Ossification, atrophy and vulnerability (Ashby / Piaget)
5
26
Implications for ‘information’
Notes:
• We can’t expect to have considered all possible hypotheses in advance.
• However, we do know when the truth is ‘something else’ because the weights of evidence are poor for the assumed hypotheses.
• Thus we can detect deception and ‘fixation’ (a form of self-deception).
5
ConclusionsSection 6
28
Conclusions
• The common concepts of information assume that systems are globally ‘simple’.
• Our systems of interest are not simple, but may be piece-wise ‘simple’.
• Jack Good’s ‘weight of evidence’ can be used to ‘bridge’ ‘islands of simplicity’.
• Using ‘weight of evidence’ gives significant ‘added value’ to using just Shannon information.
6