1400 (071) Belanger-Martel-Guitouni · (command style) • Very high likelihood for more than one...

Defence Research andDevelopment Canada

Recherche et développementpour la défense Canada Canada

Identification of Data Sets for a Robustness Analysis

Micheline Bélanger 1, Jean-Marc Martel 2, Adel Guitouni 1

1RDDC Valcartier, 2Université Laval

10th ICCRTS

Defence R&D Canada – Valcartier • R & D pour la défense Canada – Valcartier

Agenda

• Canadian Forces Operations Planning Process (CFOPP)

• Multicriteria Decision Aid (MCDA) Methodology

• Robustness Analysis

• Identification of possible data sets based on DM local preferences

• Identification of plausible data sets


InitiatingDirective

SOR

Orientation ConceptDevelopment

Political&

MilitaryAssessment

PlanDevelopment

CPG CONOPs OPLAN

Canadian OPP and Estimate Process

PlanReviewInitiation

Mission AnalysisMission Statement

COAsDevelopment

COAsRefinement

COAs War gaming

COAsComparison

Information Brief

Decision Brief

Estimate Process


Comparison of COAs

COA2

COAm

COA1

...

COA1

COA2

COAm

...

Ranking or SelectionSet of COAs

Decision Maker’sPreferences


Initial Decision Maker’s Preference: COAsEvaluation Criteria

Flexibility

Complexity

SustainabilityOptimum Use of Resources

Factors

Risk

Covering Operational Tasks

Covering Mission’s Locations

Covering Enemy’s CoAs

Operation Complexity

Logistic Complexity

C&C Complexity

Sustainability

Cost of Resources

Criteria

Impact of Sensors Coverage Gaps

Military Personnel Loss

Collateral Damages

Equipment Reliability

Personnel Effectiveness

Confrontation Risk


Comparison of COAs

COA2

COAm

COA1

...


COA1

COA2

COAm

...

Best possible compromise considering:–Conflicting evaluation criteria–DM’s values and preferences

Set of COAs Ranking or Selection


Comparison of COAs

COA2

COAm

COA1

...


COA1

COA2

COAm

...

MCDAMethodology

Set of COAs Ranking or Selection


Applying MCDA Methodology to Compare COAs

DM ’sPreferences

Criteria (1…n)C1 ... Cj ... Cn

a1 e11 ... e1j ... e1n

: : : : : :ai ei1 ... eij ... ein

: : : : : :CoA

s (1…

m)

am em1 ... emj ... emn

CoA1

CoA2

CoAm

...

MulticriterionAggregationProcedure

Local Preferences


Modelling Decision Making Styles: Introduction of Local Preferences Modelling

• Each criterion is assigned a coefficient of relative importance (πj), which might represents

– a “trade-off” or a “voting power”

• When comparing two COAs, three types of thresholds are introduced

– Indifference (qj) thresholds

– Preference (pj) thresholds

– Veto (vj) thresholds


Modelling Decision Making Styles: Introduction of Local Preferences Modelling (2)

• Indifference (qj) thresholds represent:

– the highest difference between the evaluations of two COAs, according to a given criterion j, for which the decision-maker is incapable to make a clear choice between these two alternatives, given that everything is the same otherwise

• Preference (pj) thresholds represent:

– the smallest difference between the evaluations of two alternatives, according to a given criterion j, for which the decision-maker is able to make a clear choice of one, given that everything is the same otherwise

• Veto (vj) thresholds represent:

– the smallest difference between the evaluations of two alternatives, according to a given criterion j, for which the decision-maker cannot conclude that an alternative ai is as good as ak, if the performance of akis higher than the performance of ai and if the difference of the evaluations between them is greater than νj (even if the performance of ai is higher than ak for all others criteria).


Possible Preference Relationship When Comparing two COAs

ki aa ~k i a a φ ik aa φ

j ij p e − jij pe + kje ije jij q e + jij qe −

k i φ fa a ikφfaa

jj

jkjijki

jkjijjkfji

jkjijki

qpwhere

qeeaa

peeqaa

peeaa

>

≤−⇔

<−<⇔

+≥⇔

j

j

~

φ

φ

indifference between ai and ak

strict preferenceof ai over ak

strict preferenceof ak over ai

weak preferenceof ai over ak

weak preferenceof ak over ai

DiscriminationThresholds


Problematic

• Modelling requires transforming, reduction and decomposition of the reality

– It is impossible to derive exact models of the situation

• The complexity of the military operation context prevents from deriving exact and precise values to represent Commanders preferences structures (command style)

• Very high likelihood for more than one plausible data set to represent the Decision Maker’s preferences structure

– Possibility to get more that one “optimal”solution for the same decision-making situation


Why a Robustness Analysis

• The imperfection of the data set obtained should be properly considered in decision analysis

• Robustness analysis should consider all plausible data sets in order to identify a robust ranking of plausible good decisions (COAs)

• A specific set of data instantiates a potential realization of the model of decision.


Robustness [Kouvelis and Yu, 1997]

• From the point of view of the optimality

– The solution of a mathematical program is qualified as robust, if it remains in neighbourhood of the optimum for all plausible data sets of the model

• Generalisation from optimality to best compromise.

• Since the approach of robustness is crucially based on the process of generation of plausible data sets, it requires a good knowledge of the environment in which the decision take place


Robust ranking approach proposed

• Three critical steps were identified for robustness analysis in the context of COAs comparison:

– an approach to model all the data sets that instantiate the decision-maker’s preferences, which are “not so well known”;

– a method to aggregate the pre-orders generated from each data set;

– a robustness criterion suited for the decision-making situation.


Identification of Possible Data Sets

PlausibleCoefficients of

relative importance of the Criteria

PlausiblePreference Modelling

ThresholdsFiltering Plausible

Data Sets

Plausible Data Sets

Possible Data Sets


Coefficients of relative importance of the criteria (CRIC)

• Identification of intervals

– based on decision-maker’s intervals

– based on decision-maker’s explicit values

[π1(j) ,π2

(j)] njj ,...,1,10 =<< π


CRIC Based on decision-maker’s explicit values

( ) ( ) ( )nj ggg ,...,,...,1is the least important criterionis the most important criterion ( )1g

( )ng

[ ] [ ] [ ]2)(

1)(

2)(

1)(

2)1(

1)1( ,,,,,,, nnjj ππππππ ΚΚ

1,...,2and,1,0with 1)1(

2)(

2)1(

1)(

2)1(

1)( −=∀≤≥⟨⟩ −+ njjjjjn ππππππ


CRIC Based on decision-maker’s explicit values (2)

( )( ) ( )

2

21jj

j

πππ

+=Normalized with:


Ex oequo Intervals

( )

( ) ( )

( )

( ) ( ) ( )

( )

( ) ( )[ ]

( ) ( )[ ] ( ) ( )[ ]

( ) ( )[ ]

( ) ( )[ ] ( ) ( )[ ] ( ) ( )[ ]

( ) ( )[ ]25

15

24

14

24

14

24

14

23

13

22

12

22

12

21

11

5

444

3

22

1

,

,,,,,

,

,,,

,

,,

),(

ππ

ππππππ

ππ

ππππ

ππ

g

ggg

g

gg

g

[ ] [ ] [ ]2)(

1)(

2)(

1)(

2)1(

1)1( ,,,,,,, nnjj ππππππ ΚΚ 1and0with 2

)1(1

)( ⟨⟩ ππ n

Process ex oequoas one (block)

( ) ( ) ( )nj ggg ,...,,...,1is the least important criterionis the most important criterion ( )1g

( )ng


CRIC Based on a Pre-Order of Importance

π(1) ≥ π(2) ≥ …. ≥ π(n) with πj > 0 and , ∑=

=n

jj

1

1π

Example with 15 criteria :

Consider to have 6 data setsπ1 → the c.r.i. are equally balanced π6 → the c.r.i. are decreasing from 1 to 1/n

Reduce the values by slices on a basis of

(n-2)/4


Indifference Thresholds

– DM interval– Otherwise

• DM value

• Default value

→ 80% and 60%

[ ] 0with,, 121 ≥∀ jjj qjqq

jj EXq 25.015.0' =

'jq


Preference Thresholds

– DM interval– Otherwise

• DM value

• Default value

→ 80% and 60%

[ ] jpp jj ∀21 , jjjj Epqp ≤≥ 221 and

( )jjjj qvEp −+= 05.025.0'

'jp


Veto Thresholds

– DM interval

– Otherwise

• DM value

• Default value

→ 80% and 60%

[ ] 2121 with, jjjjj pvvvv >⇒

j

jj

Ev

π

25.0=

jv


Filtering Plausible Data Sets

• To reduce the number of data sets

– For each interval, use only 3 data

• First value, middle one, last one

– Treatment of parameters as groups or blocks


Discussion

• Characteristics of the proposed military decision-making model – A=(a1,…,ai,…,am);– Λ/C=(g1,…,gj,…,gn);– E=(eij =gj(ai), i=1,...,m; j=1,....,n);– M=(πj, vj(eij),qj(eij),pj(eij), i=1,...,m; j=1,...,n);

and– a multicriterion method, PAMSSEM, within the

framework of the ranking problematic.given m alternatives and n attributes/criteria


Discussion

• Identification of possible values for:

– coefficient of relative importance (πj)

– discrimination thresholds

• indifference (qj )

• preference (pj )

– veto thresholds (vj )

• Identification of plausible data sets


Conclusion

• Robust results should be less influenced by the imperfection of the data occurring in the evaluations of the courses of actions as well as in the instantiation of parameters representing decision-maker’s preferences during the modeling process of a military situation

• Considering all plausible information that might represent the decision-making context– Not constrained to a single data set

• Robustness concept should be generalised to other information/knowledge analysis methodologies– e.g.; IPB and Enemy’s estimates

Date post:	25-May-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

1400 (071) Belanger-Martel-Guitouni · (command style) • Very high likelihood for more than one...

Documents