Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

1

Efficient Scenario Updating in Emergency Management

Tina Comes

Institute for Industrial Production (IIP)

Karlsruhe Institute of Technology (KIT)

comes@kit.edu

Niek Wijngaards

Thales Research & Technology Netherlands /

D-CIS Lab

niek.wijngaards@d-cis.nl

Frank Schultmann

Institute for Industrial Production (IIP)

Karlsruhe Institute of Technology (KIT)

frank.schultmann@kit.edu

ABSTRACT

Emergency managers need to assess, combine and process large volumes of information with varying degrees of (un)certainty. To keep track of the uncertainties and to facilitate gaining an understanding of the situation, the

information is combined into scenarios: stories about the situation and its development. As the situation evolves,

typically more information becomes available and already acknowledged information is changed or revised.

Meanwhile, decision-makers need to keep track of the scenarios including an assessment whether the infor-

mation constituting the scenario is still valid and relevant for their purposes. Standard techniques to support sce-

nario updating usually involve complete scenario re-construction. This is far too time-consuming in emergency

management. Our approach uses a graph theoretical scenario formalisation to enable efficient scenario updating.

MCDA techniques are employed to decide whether information changes are sufficiently important to warrant

scenario updating. A brief analysis of the use-case demonstrates a large gain in efficiency.

Keywords

Scenario-based reasoning, situation awareness, multi-criteria decision support, scenario management and update

INTRODUCTION

Emergency situations are characterized by their complexity and the heterogeneity of the available information

(Mendonca, Vieira and Sousa, 2007). To implement adequate mitigation measures, emergency managers must

make sense of the situation, although information may be lacking, uncertain or conflicting (Van der Walle and

Turoff, 2008). Additionally, emergency managers are confronted with redundant or irrelevant information causing information overload (Schaafstal, Johnston and Oser, 2001). Moreover, the situation itself and the in-

formation about it evolve dynamically: as time passes, new or more precise information may become available,

whereas some already acknowledged pieces of information are confirmed or proven to be false.

Scenarios, which describe the situation and how it may develop, support emergency managers in gaining situa-

tion awareness (Endsley, 1995; Wright, Cairns and Goodwin, 2009). Scenario-based decision-making usually is

supposed to start with the scenarios’ conceptualisation and to end with the usage of scenarios for decision sup-

port (Huss, 1988). The intermediate stages are scenario planning, development, simulation, analysis and evalua-

tion (Schoemaker, 1993). In this concept, scenario updating, i.e. accounting for and integrating newly available

information requires a complete re-construction of the scenarios (Ahmed, Sundaram and Piramuthu, 2010). This

is clearly infeasible in emergency situations, when time is bounded, and the availability of the experts contrib-

uting to the scenarios is limited.

To support emergency managers, we present a well-structured and efficient approach for handling newly in-coming or updated information to ensure valid, reliable and manageable scenario-based situation assessment.

This approach is efficient with respect to three aspects. First, it considers the effort needed to update the sce-

nario. In this manner, time and resource constraints can be taken into account, and the feasibility of an update

can be assessed, and impracticable scenario updates can be excluded. Secondly, a relevance assessment for the

new information is introduced. This assessment answers the question whether the updated information is – given

the emergency managers’ preferences – sufficiently important to justify the effort of an update. Thirdly, if an

update is required, the effort is reduced by enabling still valid information to be re-used and updating only those

branches of the scenario that are affected by the new information.

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

2

The scenario updating mechanisms have been researched together with practitioners from emergency manage-

ment (EM) authorities and applied to a use-case. In this use-case, a train wagon containing chlorine is damaged,

setting off a chain of events that requires public safety personnel to assess the developing situation and take

multiple possible future developments into account. This use-case is further explained and used throughout this

paper to illustrate the impact of our efficient scenario updating mechanisms.

This paper first describes our approach to distributed scenario-based sense-making in dynamic situations such as those encountered in EM. This includes the definition of scenarios and their use for sense- and decision-making,

the description of scenario construction, and the management of the construction process. The paper then de-

scribes the main challenges for using scenarios in dynamic environments; setting the stage for the description of

scenario updates. The efficient scenario update method is based on two decisions: assessing the effort for updat-

ing scenarios and assessing the relevance of new information (to justify scenario updates). The use-case pro-

vides a running example showcasing the efficient scenario update method and substantiating the gain in effi-

ciency. The paper concludes with a brief summary and an outlook on additional research topics.

DISTRIBUTED SCENARIO-BASED SENSE-MAKING IN EMERGENCY MANAGEMENT

Emergency managers need to make decisions, often with important consequences, despite stress and time pres-

sure. Decision-makers react with habitual responses or well-learned behaviours in these circumstances (Plotnick

and Turroff, 2009; Staw, Sandelands and Dutton, 1981). If the emergency situation is unprecedented or different

from training cases, the resulting decisions may be ineffective or cause further harm. In these situations it is im-

portant that emergency managers are provided all relevant information in order to assess the situation and to

make well-founded rational decisions (Plotnick and Turoff, 2009; Van der Walle and Turoff, 2008). Hence, the need for well-structured support providing and presenting the relevant information in an adequate manner arises.

Typically, emergency managers need to collaborate within emerging organisations-of-organisations. Therefore,

sense-making is of particular importance. The basic idea of sense-making is that the perception of reality is an

ongoing process that emerges from efforts to create order and to make retrospective sense of what occurs

(Weick, 1993). As the situation evolves, information is typically updated and new information becomes availa-

ble. Reasons for updates (of already existing information) may be that experts have been able to perform meas-

urements or to use more sophisticated models requiring more time than their original assessment. Furthermore,

additional experts may join the EM process, providing further information. Altogether, emergency managers are

confronted with information updates that comprise changes of existing information, the availability of additional

or the retraction of outdated information, as well as an update on the meta-information, e.g., about the uncer-

tainty of the provided information. Due to this dynamic evolution, the implemented mitigation measures often need to be quickly re-assessed and eventually even be adapted or re-planned (Shen and Shaw, 2004).

Requirements for gaining situation awareness in EM comprise the participation of experts from different do-

mains (Pavlin, Kamermans and Scafes, 2010). To avoid the problem of information overload (Fiedrich and

Burghardt, 2007) it is important that only relevant information is processed and distributed. Additionally, con-

straints in terms of expert availability and limited time need to be respected. To provide support to emergency

managers, the individual pieces of information must be combined and processed into meaningful and easily un-

derstandable descriptions (Comes, Conrado, Hiete, Kamermans, Pavlin and Wijngaards, 2010). To avoid cogni-

tive biases such as overconfidence or the availability bias (Kahneman, Slovic and Tversky, 1982), it is important

that the situation assessment conveys the (un-)certainty associated to the analyses.

To provide support in these situations, we propose following a distributed scenario construction procedure

(Comes, Hiete, Wijngaards and Schultmann, 2011). The first step in this procedure is the elicitation of the deci-

sion makers’ information needs: which information is required to gain situation awareness? These information

needs are represented by a set of typed variables FOCUS={tvF1,…, tvF

n}, where each tvFi contains information of

a specific type (e.g., text, .jpeg, mp3, symbol). The use of FOCUS enables filtering of information: only infor-

mation relevant to determine at least one of the focus variables is considered. To discover the relevant variables

and their interdependencies, a distributed approach based on the resolution of task dependencies is used, which

lets experts (humans and automated systems) define their capabilities in terms of a task they can perform and in-

formation they require to this end (Pavlin et al., 2010). To provide this additional information, the system identi-

fies and connects the experts via software agents (Pavlin et al., 2010): this achieves a loosely coupled distributed

systems, which is orchestrated (via workflows) to jointly work on a problem at hand. In this manner, a network

of experts collaborating to generate scenarios arises (Comes et al., 2010).

This network of dependencies among variables (specified by the experts) can be represented by a directed acy-clic graph (DAG), which enables distributed information processing and scenario construction (Comes et al.,

2010; 2011). To this end, the information is processed in a bottom-up manner following the directed links in the

DAG. Each expert uses his local knowledge and procedures to determine the possible state(s) of the variable tvj,

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

3

for which he agreed to provide results given the information he receives. If there is uncertainty about the value

V(tvj), an expert can pass on several possible estimates V1(tvj),…, Vn(tvj) per variable. Therefore, the number of

scenarios grows with growing number of uncertain variables and growing number of values per uncertain varia-

ble. Per value Vi, the experts are also asked to provide an assessment of the likelihood that tvj’s value in fact will

be Vi(tvj), which is represented by statusi(tvj). They are free to provide qualitative assessments (such as “highly unlikely”), probability bounds, and estimates of probabilities or to declare that the value is certain. If

they are not able to describe statusi(tvj), this ignorance is also denoted in the status by setting the probability

bounds to 0 and 1 (Comes et al., 2011). On the whole, a scenario Si can be understood as a tuple

, where STVi denotes the set of relevant variables, svi contains one value per

variable in STVi, statusi specifies the according status, and DIi represents the interdependency structure (i.e.,

directed links) of the DAG for Si. Note that FOCUS STVi.

This distributed approach in which experts define their task capabilities ensures that the experts are provided

with the very information they have judged necessary to provide their service. In this manner, the problem of in-

formation overload, which includes the provision of irrelevant or redundant information (Gonzalez and Bharosa,

2009), is reduced. Another important requirement for situation awareness, information consistency, is ensured

from the beginning onwards, under the assumption that the local assessments of the experts reflect the “state of

the art” as good as possible, given their current knowledge and the available time (Comes et al., 2010). Moreo-ver, the scenarios are coherent, as not only the variables’ values are presented to the emergency managers, but

the representation as a DAG describes their interdependencies. This coherence fosters transparency and provides

an overview on (information) dependencies.

As the scenarios need to be tailored to the emergency managers’ needs, it is important to trace and steer scenario

construction in a manner that ensures that all requirements are met as good as possible. To this end, the distrib-

uted scenario construction is combined with a (decision-centric) scenario management component. Particularly,

this allows the (potential) combinatorial explosion to be managed and ensures that a set of scenarios can be gen-

erated in due time whilst respecting the experts’ (local) constraints and requirements.

Figure 1: Configuration of expert network for the illustrative example

Figure 1 shows the arising network for the EM example. The FOCUS variables can be found on the right; further

relevant variables are shown as ovals. Variables, about whose assessment there is (inherent) uncertainty, leading

to an increase of the number of scenarios, are represented in black.

Table 1 shows a subset of (complete) scenarios (S1 to S9) constructed to gain situation awareness for the EM

use-case, using the network as shown in Figure 1. The variables with a star (*) denote the uncertain variables

that are shown to take multiple values. The other variables are certain, yet may differ in the certain value, given

values of other variables in ‘their scenario’ that these depend on. Table 1 makes explicit that a variable’s value

is not necessarily a number. The only requirement for the types of value that an expert provides is that it should

be possible to transmit it electronically and that it must be understood by the direct next experts in the DAG (connected with exactly one edge). Beyond providing an estimate of each variable, each expert also assess the

maximal duration (in minutes) of providing an estimate using his currently preferred technique, rationale or

model (line 2 in Table 1). Additionally, the effort (line 3 in Table 1) is a means for experts to specify their cur-

rent workload. The effort is assessed on a qualitative scale, where 1 corresponds to “moderate”, 2 to “busy” and

3 to “overloaded”. The experts indicating that they are “overloaded” are asked to specify the number of scenar-

ios they can at most process within a given timeframe. Here, the experts for the FOCUS variables # pp shelt &

exp, # pp unshelt & exp and firefighters indicate that they can at most handle five scenarios in two hours,

whereas the expert on police states that he can at most handle four in the same time (not shown in the table).

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

4

Table 1: Scenarios for Example use case.

tran

sfer

success

*

weath

er

leak

siz

e*

ch

em

ical

sou

rce t

erm

*

plu

me*

po

pu

lati

on

reg

istr

y

po

pu

lati

on

prese

nce*

tra

nsp

orta

tio

n

infr

ast

ru

ctu

re

bu

ild

ing

infr

ast

ru

ctu

re*

# p

p s

helt

& e

xp

# p

p u

nsh

elt

&

ex

p

fire

fig

hte

rs

[man

h]

po

lice [

man

h]

duration [min] 0 30 2 30 2 3 0 1 2 1 15 15 15 15

effort (1-3) 1 2 1 2 1 1 1 1 1 1 3 3 3 3

S1 1 NW none Cl none none 0 0 none none 0 0 0 0

S2 0 NW large Cl Big Area-big-1 2500 2000 road works standard 300 200 800 250

S3 0 NW large Cl Big Area-big-1 2500 2000 road works some dilapidated 500 200 850 300

S4 0 NW large Cl Big Area-big-2 2000 1750 standard standard 400 400 400 160

S5 0 NW large Cl Big Area-big-2 2000 1750 standard some dilapidated 700 400 450 180

S6 0 NW large Cl Medium Area-med-1 1500 1100 standard standard 800 300 160 80

S7 0 NW large Cl Medium Area-med-1 1500 1100 standard some dilapidated 500 600 250 100

S8 0 NW large Cl Medium Area-med-2 1500 1600 standard standard 850 750 250 125

S9 0 NW large Cl Medium Area-med-2 1500 1600 standard some dilapidated 750 350 340 140

FOCUSindicatorsvariables

scenario

On the basis of Figure 1, the longest path from a source to a sink node can be calculated to assess an upper bound for the duration of the scenario construction process (per scenario). In Figure 1, the edges are annotated

with the respective durations, and the longest path connecting Weather to the Emergency Managers has a

length (or duration) of 50 (minutes). For the effort, the maximum operator is used, as it is assumed that over-

loading experts should be avoided, and that the path is only as good as the weakest (or most overloaded) link. In

this example, all paths that include one of the focus variables are assigned the same effort (namely, 3). The

above set of scenarios is the base-line for our further considerations on efficient scenario updating.

INDICATOR ASSESSMENT

To facilitate the assessment of partial and complete scenarios, an indicator framework is used, where the set Indi

⊂ STVi is a subset of the variables in the scenario. Each indicator is, via the DAG, causally connected to one or

more FOCUS variables; all indicators are together connected to all FOCUS variables. The indicators are chosen

such that differences in their values are a reliable indicator for differences in values of FOCUS variables. Hence,

partial scenarios (i.e., the scenarios, for which the FOCUS variables’ values are not all known), can be assessed

regarding their relevance with respect to different FOCUS variables’ values. In Table 1, the indicator variables

which allow for drawing conclusions on the FOCUS variables are highlighted in grey.

Table 2: Indicator assessment for basic scenario set

indicator

S1 S2 S3 S4 S5 S6 S7 S8 S9 weight

building infrastructure* 1,00 0,50 0,00 0,50 0,00 0,50 0,00 0,50 0,00 0,25

transportation infrastructure 1,00 0,00 0,00 0,50 0,50 0,50 0,50 0,50 0,50 0,25

population presence* 1,00 0,00 0,00 0,13 0,13 0,45 0,45 0,20 0,20 0,50

indicator assessment 0,00 0,88 1,00 0,69 0,81 0,53 0,65 0,65 0,78

scenarioindicator

To make comparisons across indicators that are typically measured on different scales each indicator is mapped

by means of a value function onto a value in [0,1], where, corresponding to approaches of Multi-Attribute Value

Theory, 0 signifies the worst value, and 1 the best (Keeney, 1996). To determine these value functions, the di-

rection of influence of the indicator on the focus variables is determined (increasing or decreasing) and for each

focus variable, it is determined if the according preferences are decreasing (i.e., a low value is preferred, e.g,

manh, man hours) or increasing (e.g., health). By multiplying the respective assignments (each represented by +1 or -1), one determines the overall direction for the indicator. Next, the shapes of the value functions need to

be determined. Here, we assume linear functions, methods how to derive further shapes have been explained by

von Winterfeldt and Edwards (1986). Subsequently, the indicator values’ assessments are aggregated into a sin-

gle value. For its transparency and ease of use, we use a weighted sum to aggregate the single assessments (cf.

Stewart 1995). The importance of an indicator is derived from structural considerations. It is assumed that the

more FOCUS variables an indicator influences, the more important this indicator it is. Hence, the weight of each

indicator indi is calculated by summing the weights of FOCUS variables to which it is connected. As each indi-

cator by definition has an impact on at least one FOCUS variable, each weight is greater than 0. Finally, a nor-malisation of weights is performed, to ensure that the sum of weights equals one. Table 2 specifies the indicator

weights for the use-case where the underlying structure remains the same as in Figure 1.

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

5

USING SCENARIOS IN DYNAMIC ENVIRONMENTS

In EM, it is necessary to quickly gather, analyse, distribute and process information into meaningful descriptions

of the incidents whilst keeping track of the dynamic evolution of the environment (Gonzalez and Bharosa, 2009;

Plotnick and Turroff, 2009). As the present unfolds into the future, different or more accurate information about

the incident becomes available (Mahmoud et al., 2009). The existing scenarios need to be reviewed and assessed

to determine whether the currently implemented measures and strategies must be modified or changed. Hence,

to ensure situation awareness reflecting the current knowledge about the situation best, continuous revisions and

corrections of scenarios are necessary. The key issue is that the scenario updates are conducted in an efficient

and responsive manner.

In literature, however, the problem of scenario updates is hardly systematically addressed. Usually, an update

involves a complete reassessment of the scenarios (Mahmoud et al., 2009; Pascual, 1999). In the field of context

awareness, Zhu, Pung, Oliya and Wong (2011) recently published an approach that allows distributed experts to

subscribe to information about whose updated values they need to be informed if these updates change the value

more than a threshold value. This approach is, however, only of limited applicability in the field of EM. First,

emergency situations are complex and prone to non-linear developments (Helbing, 2009). Therefore, a small

change in a variable’s value can have important consequences for further parts of the system. This causes the

problem that although an event may seem negligible when viewed in isolation, it can have relevant conse-

quences, and potential threats can be ignored (Adam, 2007). Second, in this approach each expert is free to

choose if he would like to take into account the information or not. This may lead to inconsistent scenarios.

A scenario can be updated due to a change in its dependency structure (represented by the DAG), a change of a variable’s value or its status. In the context of this paper, we focus on an update of values (or the likelihood as-

sessment that a variable takes a value). The dependency structure (underlying DAG) can vary as well (e.g., as

more complex networks can be applied when more time is available), but the DAG is assumed to be stable here

for simplicity’s sake. Efficient scenario updating has two aspects. First, it is necessary to assess if the new in-

formation is “relevant enough” to justify the updating effort. Both concepts (relevance and effort) are defined

rigorously in the following section enabling the formulation of the updating decision in terms of a constrained 3-

dimensional optimisation problem. Second, efficient updates allow for information to be reused that has already

been assessed and that is still valid, while updating the pieces of information whose value has changed.

EFFICIENT SCENARIO UPDATES

As scenario updating is closely related to timing issues, from now on the scenario is be annotated with the time t

when it was determined: Si = Sti = STVi , svt

i , statusti , DIi . (STVi and DIi are assumed to be constant: the DAG

is stable for these scenarios.) If an expert provides a new value at time t+T, Vt+Ti(tvj) for a variable tvj∈STVi, a

decision must be made, if the scenarios Si, which assumed Vi(tvj)= Vti(tvj) need to be updated, resulting in the

set of new scenarios SSnew. To this end, the new information’s relevance is assessed. If the information is judged sufficiently important, an efficient update needs to be employed. The efficiency gain is illustrated later.

First and foremost, the effect of scenario updating is shown for the example use-case. After having completed

the scenarios S1 to S9, shown in Table 1, the expert on the Source Term has invested time to make more accu-

rate estimations on the outflow of the Chlorine, based on pictures of the actual hole sent by the local incident

commander. The expert revokes the value “Big” (i.e., changes the status to ‘irrelevant’), and replaces the value

‘Medium’ with two possible outflow rates: 4 kg/s and 5 kg/s. The result is that immediately scenarios S2,3,4,5

are ‘pruned’. Furthermore the resulting changes to scenarios S6-S9 are shown, where the white background sce-

narios are not sufficiently different to be actually continued (hence the ‘no value’) indication for the associated

FOCUS variables (except for scenario S9, for which these values were already determined, and no updating was necessary). The reasons for completing or not completing, new scenarios are described below.

Feasibility of Performing a Scenario Update

The update of information may concern both FOCUS complete and incomplete scenarios, in which there are

variables that have not been assigned a value yet. The scenario recipients must balance the time (still) available

for the purpose at hand; the time and effort necessary for that update along with the workload of the experts in-

volved; and the (projected) effect of the new value(s) on the FOCUS variables. In case a full update of the sce-

narios cannot be accomplished (e.g., as some experts are hardly available, or as some assessments are too time

consuming), a decision must be made whether: (a) the original (consistent) scenarios are retained as a basis for

the purpose at hand, (b) a partial update (that may lead to inconsistencies) is performed by updating only those

branches of the scenarios for which an update is feasible, or (c) the most relevant scenarios are selected for up-

dating (including a recalculation of all values that are affected by the update).

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

6

For the assessment of the time and effort necessary to perform a value update, all steps within the scenario con-

struction and analysis need to be respected. Therefore, the time and effort depends on the number of scenarios Si in SSnew that need to be updated, the number of variables tvj in each STVi, whose value must be updated, and the

effort and duration these updates require.

For our efficient scenario updating method we exploit the directed acyclic graph (DAG) representing the infor-mation within a scenario: standard graph analysis concepts are now available, including a path. For assessing

the duration, each edge in the DAG is weighted with duration. To establish a bound for the maximum duration

to conduct scenario updates, the longest paths between the modified variable and any FOCUS variable and the

“bottleneck variables”, whose assessment takes the longest, need to be determined. The sum of these annota-

tions in a path provides the overall duration per path. Furthermore, the time for a complete update is larger or

equal to the time for each single scenario update. This upper bound for the duration of scenario updating can be

compared against the time remaining for the decision problem at hand taking into account time already spent on

this part of the efficient scenario updating method. For constructing 8 new partial scenarios (assuming that the

source term is already calculated): an upper bound of the duration is 5+8*15=125 min. (2:05). Note that sce-

narios are not constructed from scratch; we re-use values from the original scenarios. Assuming that the time available for the updates is 2 hours, it is clear that then at most seven scenarios can be updated. Additionally

constraints given by the experts, who can become, or already are overloaded, need to be taken into account. For

example, as the expert determining the variable police manh can at most process four scenarios in two hours, no

more than 4 scenarios can be updated within the available time.

Information Relevance Assessment

Given enough time available to conduct scenario updating, the impact of the information update needs to be as-

sessed with regard to FOCUS variables and their specific values. This impact assessment is achieved by not

computing all complete scenarios based on the information update, but rather by using the indicator framework.

For the updated information, we need to establish the (new) values only for the (causally related) indicators.

This means that a number of (new) partial scenarios must be constructed until the point that the values of the

causally related indicator variables become known; the other values of the indicators can be ‘copied’ from the

original scenarios. Our method thus re-uses values of variables that have not changed (based on the DAG struc-

ture), thereby avoiding unnecessary reassessment: only those variables that causally connect the updated varia-

ble with the indicator variable must be re-assessed. This minimal scenario updating effort must be spent, using a certain duration, to be able to assess the impact of the information update and whether some scenarios can be

pruned (including ‘old complete’ scenarios and these new partial scenarios) and which of these new partial sce-

narios can be continued until they are complete(d).

Table 3: Updating scenarios S1-S9. Scenarios S,3,4,5 are pruned, white-background scenarios are not completed.

tran

sfer

success

*

weath

er

leak

siz

e*

ch

em

ical

sou

rce t

erm

*

plu

me*

po

pu

lati

on

reg

istr

y

tran

spo

rtati

on

infr

ast

ructu

re

po

pu

lati

on

prese

nce*

bu

ild

ing

infr

ast

ru

ctu

re*

# p

p s

helt

& e

xp

# p

p u

nsh

elt

&

ex

p

fire

fig

hte

rs

[man

h]

po

lice [

man

h]

duration [min] 0 30 2 30 4 3 0 1 2 1 15 15 15 15

effort (1-3) 1 2 1 2 3 1 1 1 1 1 3 3 3 3

S1 1 NW none Cl none none 0 none 0 none 0 0 0 0

S'6a 0 NW large Cl 4 kg/s Area-med-1a 1500 standard 850 standard 600 250 160 100

S'6b 0 NW large Cl 5 kg/s Area-med-1b 1500 standard 1100 standard - - - -

S6 0 NW large Cl Medium Area-med-1 1500 standard 1100 standard 800 300 160 80

S'7a 0 NW large Cl 4 kg/s Area-med-1a 1500 standard 950 some dilapidated - - - -

S'7b 0 NW large Cl 5 kg/s Area-med-1b 1500 standard 1050 some dilapidated - - - -

S7 0 NW large Cl Medium Area-med-1 1500 standard 1000 some dilapidated 500 500 250 100

S'8a 0 NW large Cl 4 kg/s Area-med-2a 1500 standard 1500 standard 1000 500 250 120

S'8b 0 NW large Cl 5 kg/s Area-med-2b 1500 standard 1400 standard - - - -

S'8 0 NW large Cl Medium Area-med-2 1500 standard 1600 standard 850 550 350 125

S'9a 0 NW large Cl 4 kg/s Area-med-2a 1500 standard 1700 some dilapidated 1000 700 350 120

S'9b 0 NW large Cl 5 kg/s Area-med-2b 1500 standard 1800 some dilapidated 950 750 400 120

S9 0 NW large Cl Medium Area-med-2 1500 standard 1600 some dilapidated 750 350 340 140

scenario

variables FOCUSindicators

The example in Table 3 and the DAG in Figure 1 show that the update of information on the source term causes

a change in values for the indicator population presence via the variables plume and population registry. Sce-

narios S2 to S5 can be pruned immediately, as they assumed a “Big” source term, and therefore relied on wrong

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

7

assumptions. Table 3 shows that the indicator transportation infrastructure is not significant any more: there are

no differences in the values for all valid scenarios. Therefore, transportation infrastructure is deleted from the

set of indicators. Moreover, Table 3 shows that these new values for the indicators (available in all updated par-

tial scenarios) can be aggregated-compared against the old values of this indicator (in the original (complete)

scenarios). Table 4 specifies the re-assessed weights for the remaining indicators.

Table 4: Indicator assessment for updated scenarios compared to original assessment

indicator

updated indicator S'1 S'6a S'6b S'7a S'7b S'8a S'8b S'9a S'9b weight

population presence 1,00 0,58 0,45 0,53 0,38 0,25 0,30 0,15 0,10 0,67

building infrastructure 1,00 0,5 0,5 0 0 0,5 0,5 0 0 0,33

evaluation of indicator 1,00 0,45 0,53 0,65 0,75 0,67 0,63 0,90 0,93

difference 0,00 0,03 0,05 0,30 0,65 0,32 0,28 0,67 0,70

1,00

S1

updated scenario

original scenario

original evaluation of

indicator S6 S7 S8 S9

0,48 0,35 0,35 0,23

Table 4 shows the recalculated indicators and their difference with respect to the original evaluation of the indi-cator (see also see Table 2). In Table 4 the scenarios are highlighted in which the difference of the indicator is

large enough to warrant further completion of the scenario and the other scenarios are not further completed.

Table 5: Complete scenarios for scenario S6.

tran

sfer

success

*

weath

er

leak

siz

e*

ch

em

ical

sou

rce t

erm

*

plu

me*

po

pu

lati

on

reg

istr

y

po

pu

lati

on

prese

nce*

tra

nsp

orta

tio

n

infr

ast

ru

ctu

re

bu

ild

ing

infr

ast

ru

ctu

re*

# p

p s

helt

& e

xp

# p

p u

nsh

elt

&

ex

p

fire

fig

hte

rs

[man

h]

po

lice [

man

h]

duration [min] 0 30 2 30 4 3 0 1 2 1 15 15 15 15

effort (1-3) 1 2 1 2 3 1 1 1 1 1 3 3 3 3

S'6a 0 NW large Cl 4 kg/s Area-med-1a 1500 standard 850 standard 600 250 160 100

S'6b 0 NW large Cl 5 kg/s Area-med-1b 1500 standard 1100 standard 800 300 160 100

S6 0 NW large Cl Medium Area-med-1 1500 standard 1100 standard 800 300 160 80

scenario

variables indicators FOCUS

The example in Table 5 shows (with all three scenarios completed for illustrative purposes), when e.g. looking

at the scenarios S’6a and S’6b, which are ‘new’ scenarios related to scenario S6, that scenario S’6b is not different

from scenario S6 – for the value of the indicator ‘Population Presence’. Hence, scenario S’6b does not need to

be continued, and the original scenario S6 can be retained. However, scenario S’6a is sufficiently different from

scenario S6 for this indicator, and warrants further construction into a complete scenario (shown in Table 3).

In sum, only those new partial scenarios are further constructed into complete scenarios, if the value of the indi-cator assessment is sufficiently different from the value of original assessment. In this manner, the indicator as-

sessment can be interpreted as a similarity measure. The threshold for determining if the scenarios are “suffi-

ciently diverse” to justify an update is currently set (per default, adjustable by the decision maker) at 0.05: be-

low that value scenarios are ‘sufficiently similar’. Furthermore, before completing any of these partial scenarios,

the time and effort requirements need to be respected.

The amount of differences in values for the indicators is related to a ranking of feasible updates. This ranking

takes into account the expected impact of the change in value of the indicator, the duration bounds and effort

bounds. The ranking of feasible updates is determined via solving a constrained 3-dimensional optimisation

problem, in which the feasible updates are considered the alternatives, and the goal function to be minimised has

the three dimensions impact, duration and effort, where the preferences of the decision maker are elicited to in-

dicate relative importance of these criteria (per default: minimum duration and minimum effort and maximum impact are preferred), see the call-out box below. The resulting ranking is used in, e.g., situations when limited

time is available and a selection of partial scenarios for updating needs to be made.

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

8

Decision on Scenario Update Due to a Change of Value. Assume that new information on J independent variables tvj1, …, tvjJ ∩iI STVi

is available. A decision must be made which of the values are updated. Below, only the update of single variables’ values is investigated, to

rank the most useful updates. This optimisation problem is defined as follows, where the updating of the variables tvjk is denoted ak, and not

performing an update is denoted a0: w1v1D(x1k(aj)) + w2v2D(x2k(aj)) max!

Subject to: j = 0,…,J wl ≥ 0 (l=1,2) w1 + w2 = 1 dur(PLi,tvj) ≤ Tmax(Pupω) – Tanalysis

Here, x1k(aj) denotes the score of the duration of effort assessment (depending on the information available), while x2k(aj) denote the result

of the impact assessment (depending on the purpose) for j=1,…J. For j=0 (i.e., the option of not performing any update), one defines

x1k(a0)=x2k(a0)=0. v1D and v2D are the respective value functions modelling the decision makers’ intra-criteria preferences. Here, the prefer-

ences for the effort are assumed to be decreasing (i.e., the lower the effort the better) while the preferences for the impact are supposed to be

increasing (i.e., the bigger the difference, the more important it is to perform an update). The weights w1,w2 indicate the respective im-

portance of the objectives. The duration of the value updated is expressed in terms of the paths (see main text), and is set against the time

available for decision making minus the time needed for this analysis. In this manner, a ranking of feasible updates can be achieved.

Changes to the status of a value are also considered. E.g., direct pruning can be employed for the information

update on the Source Term, where the value Big has now the status irrelevant. This immediately implies that all

scenarios in which Source Term has the value Big can now be pruned: this is shown in Table 4 in which

scenarios S2,3,4,5 are pruned, and thus are not used for situation assessment or decision-making.

The description and example above describe the efficient scenario updating of one variable whereas our method

is defined to handle efficient scenario updating for information updates of multiple variables. The formalization

of our method is too lengthy to be described in this paper, but is described in detail in (Comes, 2011).

DISCUSSION

The impact of our efficient scenario update method can be easily shown by comparing it with complete scenario

re-construction (as is often advocated, see above) using in both cases the same metrics for duration. The effort estimation is not present in the other approaches. The efficiency of our method is related to avoiding the re-as-

sessment of information, which is still a sufficiently valid basis for the decision at hand. This includes making a

decision if the updated information justifies the effort and duration of updating. If the latter question is answered

positively, the graph theoretical formalisation facilitates re-using information that is not affected by an update.

For instance, the update of the variable Source Term does not require that the experts providing information on

the Chemical or the Weather need to provide their information again. This is different from discursive scenario

construction techniques, where interrelations are not made explicit and experts from all relevant domains are

brought together to construct the scenarios.

Furthermore by the careful construction of new partial scenarios up to the point that the relevant indicator varia-

bles obtain a (new) value, we can limit the re-assessment to only those of the partial scenarios that are consid-

ered to be sufficiently different with respect to the FOCUS variables. For ease of this comparison, we assume that both our efficient method as well as other methods immediately prune scenarios that have become obvi-

ously irrelevant because of the information update (cf. scenarios S2,3,4,5 in our example). The example shown in

Table 1 and Table 3 provides a means to exemplify this gain in efficiency:

For complete re-assessment, 8 new scenarios are constructed (S’6a-b, S’7a-b, S’8a-b and S’9a-b).

With our efficient scenario updating method, 8 partial scenarios are constructed until the point that the

indicator Population Presence has been given (new) values, re-using non-causally related values from

their ‘original’ scenarios. The variables Transfer Success, Weather, Leak Size and Chemical remain un-

changed and are adopted into the new scenarios. Then, out of these 8 new partial scenarios, only 4 are com-

pleted (S6a, S8a and S9a-b).

This is further illustrated by a calculation of the duration spent, where we assume the duration as in Table 3.

The longest path through the network has a length of 90 minutes, thus an upper bound for the duration of

each scenario construction process is 1:30 h. As the longest process within the map takes at most 40 minutes, an upper bound for the total duration of scenario construction (assuming efficient and immediate

processing of information) is 30+8*40+3+2+15=370 minutes (or 6:10 h) for complete re-assessment.

For our efficient scenario updating, the duration spent is:

For constructing 8 new partial scenarios (assuming that the source term is already calculated): an upper

bound of the duration is 5+8*15=125 min. (2:05). Note that scenarios are not constructed from scratch;

we re-use values from the original scenarios.

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

9

For completing the four selected scenarios: an upper bound of the duration 5+4*15=25 minutes.

The total duration for efficient scenario updating is 2:05h + 0:25h = 2:30h.

The actual performance gain shown above is indicative only for this small example. In more complex situations,

the performance gain depends on the variables, for which information is updated, the duration of this update, the

structure of the DAG, the variables connecting the updated variables and the related indicators, the variables

connecting the related indicators and focus variables, and the actual values produced by the involved experts, their actual time spent, and the differences of the indicator variables’ values. Nevertheless, we can guarantee

efficient updating given these constraints.

CONCLUSION

The challenge addressed in this paper is making sense of a dynamic situation while handling updates of infor-

mation and respecting time and effort constraints. We have shown that our formalised and well-structured ap-

proach to scenario-based sense-making enables efficient scenario updates. The formal graph-based approach for

scenario representation has the advantage that (direct and indirect) dependencies are made explicit. In this man-

ner, efficient scenario updating is facilitated, as updating does not require a complete revision of all scenarios,

but allows for adapting only the parts of the scenarios that are affected by the changed information.

Our solution includes an MCDA-based tool to assess the relevance of updated information, together with an

estimation of the duration and effort required to conduct the prioritised scenario updates. By exploiting the DAG

representation and the annotations provided by experts, graph theoretical techniques can be employed to derive

bounds for duration and effort. This resolves the scenario updating decision problem to a constrained 3-dimen-

sional optimisation problem. Furthermore, the use of a DAG facilitates the re-use of information that is unaf-fected by information updates and renders further efficiency gains possible. Our approach is substantiated by an

implementation that accompanies a distributed system that constructs scenarios. Our efficient scenario updating

method is available for further testing, and an additional method is being researched for updating scenarios due

to changes in the structure of the underlying graph.

Our approach is strongly related to the quality of information on the network and the experts therein: the more

knowledge available on the experts, and their reasoning (duration, effort and sensitivity to updates); the faster

and more accurate the update decisions can be made, the more efficiency can be gained. Yet, scenario updating

requires time and input from the experts involved. Usually, there is a trade-off between effort of the update

(with respect to time and expertise required) and consistency (including value, status and observation con-

sistency). This leads to another decision problem: the decision on which partial updates (i.e., only some of the

scenarios are updated) to allow, while minimizing the potential inconsistencies that may arise (as now not all scenarios are based on updated information). This is one aspect of our continuing research. Furthermore, the

generalizability to apply our approach in sense-making and decision-making tools is of interest.

ACKNOWLEDGMENTS

The authors would like to thank their colleagues at their institutes for the stimulating discussions on scenario-

based sense-making. The work described in this paper has been conducted within the Diadem project, which is

funded by the European Union under the Information and Communication Technologies (ICT) theme of the 7th

Framework Programme for R&D, ref. no: 224318, www.ist-diadem.eu.

REFERENCES

1. Adam, N., Janeja, V., Paliwal, A., Shafiq, B., Ulmer, C., Gersabeck, V., Hardy, A., Bornhoevd, C. and

Schaper, J. (2007) Approach for Discovering and Handling Crisis in a Service-Oriented Environment, IEEE

Intelligence and Security Informatics, 2007, 16 -24

2. Ahmed, D. M., Sundaram, D. and Piramuthu, S. (2010) Knowledge-based scenario management – process

and support, Decision Support Systems, 49, 4, 507–520.

3. Comes, T., Conrado, C., Hiete, M., Kamermans, M., Pavlin, G. and Wijngaards, N. (2010a) An intelligent

decision support system for decision making under uncertainty in distributed reasoning frameworks, Proc. of

the 7th International Conference on Information Systems for Crisis Response and Management, Seattle.

4. Comes, T., Hiete, M., Wijngaards, N. and Schultmann, F. (2010b) Enhancing Robustness in Multi-Criteria

Decision-Making: A Scenario-Based Approach, Proceedings of the 2nd International Conference on Intelli-

gent Networking and Collaborative Systems, Thessaloniki.

Comes et.al. Efficient Scenario Updating in Emergency Management

Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.

10

5. Comes, T. (2011) Decision Maps for Distributed Scenario-Based Multi-Criteria Decision Support.

Dissertation, Karlsruhe. URN: urn:nbn:de:swb:90-252145.

6. Comes, T., Hiete, M., Wijngaards, N. and Schultmann, F. (2011) Decision Maps: A framework for multi-

criteria decision support under severe uncertainty, Decision Support Systems, 52, 1, 108–118.

7. Endsley, M.R. (1995) Toward a theory of situation awareness in dynamic systems, The Journal of the Hu-

man Factors and Ergonomics Society, 37, 32–64.

8. Fiedrich, F. and Burghardt, P. (2007) Agent-based systems for disaster management, Communications of the

ACM, 50, 3, 41–42.

9. Gonzalez, R.A. and Bharosa, N. (2009) A Framework Linking Information Quality Dimensions and

Coordination Challenges during Interagency Crisis Response, Proceedings of the 42nd Hawaii International

Conference on System Sciences, Hawaii.

10. Helbing, D. (2009) Managing Complexity in Socio-Economic Systems, European Review, 17, 423–438.

11. Huss, W.R. (1988) A move toward scenario analysis, International Journal of Forecasting, 4, 3, 377–388.

12. Kahneman, D., Slovic, P. and Tversky, A. (1982) Judgment under Uncertainty: Heuristics and biases, Cam-bridge University Press, Cambridge.

13. Keeney, R.L. (1996) Value-focused thinking: Identifying decision opportunities and creating alternatives,

European Journal of Operational Research, 92, 3, 537-549

14. Mahmoud, M., Liu, Y., Hartmann, H., Stewart, S., Wagener, T., Semmens, D., Stewart, R., Gupta, H., Do-minguez, D., Dominguez, F., Hulse, D., Letcher, R., Rashleigh, B., Smith, C., Street, R., Ticehurst, J., Twe-

ry, M., van Delden, H., Waldick, R., White, D. and Winter, L. (2009) A formal framework for scenario de-

velopment in support of environmental decision-making, Environmental Modelling & Software, 24, 7, 798–

808.

15. Mendonca, L., Vieira, S. and Sousa, J. (2007) Decision tree search methods in fuzzy modeling and

classification, International Journal of Approximate Reasoning, 44, 2, 106–123.

16. Pascual, R. (1999) Tools for capturing and training shared understanding in teams, International Conference

on Human Interfaces in Control Rooms, Cockpits and Command Centres, Bath.

17. Pavlin, G., Kamermans, M. and Scafes, M. (2010) Dynamic Process Integration Framework: Toward Effi-

cient Information Processing in Complex Distributed Systems, Informatica, 34, 477–490.

18. Plotnick, L. and Turroff, M. (2009) Mitigating Maladaptive Threat Rigidity Responses to Crises, in: van der

Walle, Turoff, Hiltz (eds) Information Systems for Emergency Management, Armonk.

19. Schaafstal, A.M., Johnston, J.H. and Oser, R. L. (2001) Training teams for emergency management,

Computers in Human Behavior, 17, 5-6, 615–626.

20. Schoemaker, P.J.H. (1993) Multiple Scenario Development: Its Conceptual and Behavioral Foundation,

Strategic Management Journal, 14, 3, 193–213.

21. Shen, S.Y. and Shaw, M.J. (2004) Managing Coordination in Emergency Response Systems with Infor-

mation Technologies, Proceedings of the Tenth Americas Conference on Information Systems, New York.

22. Staw, B.M., Sandelands, L.E. and Dutton, J.E. (1981) Threat Rigidity Effects in Organizational Behavior: A

Multilevel Analysis, Administrative Science Quarterly, 26, 4, 501–524.

23. Stewart, T. (1995) Simplified approaches for multicriteria decision making under uncertainty, Journal of

Multi-Criteria Decision Analysis, 1995, 4, 4, 246-258

24. Van der Walle, B. and Turoff, M. (2008) Decision support for emergency situations, Information Systems

and E-Business Management, 6, 3, 295–316.

25. von Winterfeldt, D. and Edwards, W. (1986) Decision Analysis and Behavioral Research, Cambridge

University Press, Cambridge.

26. Weick, K.E. (1993) The Collapse of Sensemaking in Organizations: The Mann Gulch Disaster, Administra-

tive Science Quarterly, 38, 4, 628-652.

27. Wright, G., Cairns, G. and Goodwin, P. (2009) Teaching scenario planning: Lessons from practice in aca-

deme and business, European Journal of Operational Research, 194, 1, 323–335.

28. Zhu, J., Pung, H.K., Oliya, M. and Wong, W.C. (2011) A context realization framework for ubiquitous

applications with runtime support, IEEE Communications Magazine, 49, 9, 132–141.