Post on 05-Jul-2020
transcript
UNCERTAINTY QUANTIFICATIONWORKING GROUP
Mark AndersonExperiment & Diagnostic Design GroupDynamic Experimentation DivisionLos Alamos National Laboratory
June 28, 2001
A Definition ofSimulation Uncertainty
&A View of Total Uncertainty
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-2
PART 1—A DEF’N OF SIMULATION UNCERTAINTY
■ Why do we care?■ What is it?■ How does it apply to our models?■ What technologies are available?■ What technologies are being developed?■ What is the path forward?
Overview
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-3
WHY DO WE CARE ABOUT UNCERTAINTY?
■ Science-based stockpile stewardship requires data and models● Test measurements● High-fidelity physics-based models (FEM, etc.)● Low-fidelity physics-based models (SDOF, etc.)● Surrogate models
■ Decisions will be based on our model predictions● Safety● Security● Economic● Military
■ Accuracy and robustness is crucial to acceptance■ Accuracy/robustness ⇒ quantified uncertainties
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-4
WHAT IS UNCERTAINTY?
■ Aleatoric uncertainty (also called Variability)● Inherent variation● Irreducible
■ Epistemic uncertainty (also called simply Uncertainty)● Potential deficiency● Lack of knowledge● Reducible?
■ Prejudicial uncertainty (also called Error)● Recognizable deficiency● Bias● Reducible
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-5
HOW DOES IT APPLY TO OUR MODELS?
■ Sources of uncertainty● Measurements
– Noise– Resolution,– Quantization– Processing
● Mathematical models– Equations– Geometry– BCs/ICs– Inputs– Deterministic chaos
● Numerical models– Weak formulations– Discretizations– Approximate solution algorithms– Truncation and roundoff
● Surrogate models– Approximation error– Interpolation error– Extrapolation error
● Model parameters
■ Phases of the modeling process
Observation of Nature
Conceptual Modeling
Mathematical Modeling
Numerical Modeling
Numerical Implementation
Numerical Evaluation
Surrogate Modeling
Surrogate Implementation
Surrogate Evaluation
Valid
atio
n &
Unc
erta
inty
Qua
ntifi
catio
n
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-6
A SIMPLE EXAMPLE■ Measurement uncertainty
● Forcing function & ICs● Response
■ Math model uncertainty
● Equation form● Forcing function & Ics● Sensitive dependence on ICs
■ Numerical solution uncertainty● Integration algorithm● Time step (discretization)
■ Parameters
“Truth” Model
m
x( )tF
( ) 3xkxf ε+=
( )( ) ( )
00
3
0,0 xxxxtFxxkxm&&
&&
===++ ε
( )( ) ( ) 00
ˆ0,ˆ0
ˆ
xxxxtFkxxm&&
&&
==
=+
km ˆ,ˆ
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-7
Total Uncertainty
Parameters
DataModel
Solution
WHAT TECHNOLOGIES ARE AVAILABLE?■ Data
● Calibration w.r.t. conventional standards● Noise characterization● “Similar” or “inverse” signal processing
■ Mathematical models (Not much!)■ Numerical models
● Bounds for discretization errors● Bounds for approximate solution techniques● Bounds for truncation/roundoff errors
■ Surrogate models● DOE● Residual analysis● ANOVA
■ Parameters● Sensitivity analysis● Monte Carlo● Reliability methods (FORM, SORM, AMV, AMV+, FPI)● Fuzzy set & interval propagation methods● Stochastic FEM
■ Total uncertainty
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-8
GENERIC VIEW OF TOTAL UNCERTAINTY
■ Generic class of model-test pairs
■ Normalized comparisons
■ Uncertainty propagation
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
101
102
100
101
102
103
104
Frequency (Hz)
Acc
eler
atio
n/F
orce
Node 4 Z−Acceleration / 4 X−Force
ModelModel +/− 2 σTest
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-9
■ Measure theoretic methods● Probability theories—frequentist, Bayesian, Koopman-Carnap● Dempster-Schafer theory● Possibility theory
■ Set theoretic methods● Fuzzy set theories—classical, grey, intuitionistic, rough● Interval arithmetic● Convex sets & convex modeling
■ Dynamical systems methods● Strange attractor theory● Liapunov exponents● Complexity theory
WHAT TECHNOLOGIES ARE BEING DEVELOPED?
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-10
PROBABILITY THEORY V. DST
■ Probability theory—Based on classical measure theory (additivity)
■ Dempster-Schafer theory—Based on fuzzy measure theory (monotonicity & semicontinuity)
[ ]( )( )
( ) ( ) ( )( ) ( )
( ) ( ) ( )( ) ( )UL
UI
IL
IU
ii
n
kjkj
ii
ii
ii
n
kjkj
ii
ii
X
A
AAAA
A
AAAA
X
Pr1
PrPrPr
Pr1
PrPrPr
1Pr0Pr
1,02:Pr
1
1
+
<
+
<
−++
∑−∑=
−++
∑−∑=
==∅→ [ ] [ ]
( ) ( )( ) ( )
( ) ( ) ( )( ) ( )
( ) ( ) ( )( ) ( )UL
UI
IL
IU
ii
n
kjkj
ii
ii
ii
n
kjkj
ii
ii
XX
A
AAAA
A
AAAA
XX
Pl1
PlPlPl
Bel1
BelBelBel
1Pl1Bel0Pl0Bel
1,02:Pl1,02:Bel
1
1
+
<
+
<
−++
∑−∑≤
−++
∑−∑≥
===∅=∅→→
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-11
PROBABILITY THEORY V. POSSIBILITY THEORY
■ Probability theory—Based on classical measure theory (additivity)
■ Possibility theory—Based on fuzzy measure theory (semicontinuity)
[ ]( )( )
( ) ( ) ( )( ) ( )
( ) ( ) ( )( ) ( )UL
UI
IL
IU
ii
n
kjkj
ii
ii
ii
n
kjkj
ii
ii
X
A
AAAA
A
AAAA
X
Pr1
PrPrPr
Pr1
PrPrPr
1Pr0Pr
1,02:Pr
1
1
+
<
+
<
−++
∑−∑=
−++
∑−∑=
==∅→ [ ] [ ]
( ) ( )( ) ( )
( ) ( )
( ) ( )iii
i
iii
XX
AA
A
XX
NecinfNec
APossupPos
1Nec1Pos0Nec0Pos
1,02:Nec1,02:Pos
i
=
=
===∅=∅→→
I
U
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-12
POTENTIAL UNCERTAINTY METRICS■ Hartley measure for nonspecificity
■ Generalized Hartley measure for nonspecificity in DST
■ U-uncertainty measure for nonspecificity in possibility theory
■ Shannon entropy for total uncertainty in probability theory
■ Generalized Shannon entropy for total uncertainty in DST
■ Hamming distance for fuzzy sets
( ) AAAAH ofy cardinalit is ,log2=
( ) ( ) [ ] ( ) ( ) 1,0,1,02:,log22
2 =∑=∅→∑=∈∈ XX A
X
AAmmmAAmmN
( ) ( ) ( )∑−=∈Xx
xpxppS 2log
( ) ( ) ( ) { }( ) irrxxrirrrU ii
n
iii ∀≥=∑ −=
+=
+ 12
21 ,Pos,log
( ) ( ) ( ) X
Axx
Xxxxp
ApABelppAUx
2,logmaxBel 2 ∈∀∑≤∑−=∈∈
( ) ( )[ ] ( ) function membership is ,121 xAxAAfXx∑ −−=∈
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-13
WHAT IS THE PATH FORWARD?
■ Some type of uncertainty quantification is required■ Salient points
● Measure predictive capability ⇒ Compare data & predictions● Experiments should be designed to facilitate comparisons● “Adequate” quantification of predictive capability ⇒ lots of data● Interpolation/extrapolation beyond observations ⇒ inference
■ No comprehensive framework/toolbox exists● Application dependent● Different types of uncertainty require different tools
■ Hypothetical approach● Characterize measurement uncertainty● Characterize/propagate parametric uncertainty● Bound/propagate solution uncertainty● Estimate (generically) total uncertainty● “Subtract” to estimate model uncertainty
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-14
PART 2—A VIEW OF TOTAL UNCERTAINTY
■ What is total uncertainty?■ Prototypical application: linear structural dynamics
● Methodology● Example: Space truss structure
■ Generalization to arbitrary applications● Methodology● Example: Nose cone crushing● Example: Blast response of R/C wall
■ Conclusions
Overview
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-15
WHAT IS TOTAL UNCERTAINTY?
■ Total uncertainty is simply a measure of the difference between experimental data and model predictions
■ Practical considerations● There rarely exists enough samples for a given simulation scenario● Simple differencing leads to “small differences of large numbers”
■ A candidate approach● Consider “generic classes” of test-analysis comparisons● Normalize information so that differences are “perturbations”
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-16
PROTOTYPICAL APPLICATION: LINEAR DYNAMICS
■ Classical normal modes
■ Modal mass and stiffness matrices
■ Assumed “true” modal mass and stiffness matrices
■ Normalization of test modes
■ Cross-orthogonality of analysis and test modes
■ Differences between analysis and test modes
( )( ) test0
analysis00000
K
K
=−=−
φλφλ
MKMK
λφφφφ
00000
0000
==
==
KkIMm
T
T
kkkkmImmm
∆+=∆+=∆+=∆+=
λ00
0
IMT =φφ 0
ψφψφφφφψφ
==
=00000
0 assumedMM TT
K
( ) ψφψφφφφλλλ
∆=−=−=∆−=∆
000
0
I
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-17
PROTOTYPICAL APPLICATION (CONT’D)
■ Structure-specific covariance matrix of uncertainty (biased)
■ Generic covariance matrix of total uncertainty (unbiased)
■ Propagate thru model● Linear covariance propagation● Interval propagation● Monte Carlo simulation
■ Normalized modal metrics for total uncertainty quantification
■ Vectorization of normalized differences
( )( )ζζζ
λλψψλλλ
ψψ
0
21000210~
−=∆∆−∆−∆=∆
∆+∆−=∆−− T
T
k
m
( )( )( )
∆∆∆
=∆ζvec
~vecvec
~ km
r
[ ]( )( )[ ]T
rrrr
r
rrESrE
~~~~
~
~~~
∆∆
∆
−∆−∆=
∆=
µµµ
[ ]Trr
r
rrES ~~assumed0
~~
~
∆∆==
∆Kµ
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-18
EXAMPLE: SPACE TRUSS STRUCTURE
● Force input at Nodes 4 and 7● Acceleration response measured at Nodes 1 through 32
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
X
Y
Z
Fixed end
Free end
NASA/LaRC 8-bay truss structure
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-19
STRUCTURE-SPECIFIC VS. GENERIC VARIATIONStructure-specific variability
5 4 32
1 12
34
50
0.1
0.2
0.3
0.4
0.5
RMS
Erro
r
Mode No.Mode No.
Modal Mass
5 43
21 1
23
45
0
0.1
0.2
0.3
0.4
0.5
RM
S Er
ror
Mode No.Mode No.
Normalized Modal Siffness
54
32
1 12
34
50
0.1
0.2
0.3
0.4
0.5
RM
S Er
ror
Mode No.Mode No.
Modal Mass
54
32
1 12
34
50
0.1
0.2
0.3
0.4
0.5
RMS
Erro
r
Mode No. Mode No.
Normalized Modal Stiffness
Generic class variability
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-20
PREDICTIVE ACCURACY FOR SPACE TRUSS
101
102
100
101
102
103
104
Frequency (Hz)
Acc
eler
atio
n/F
orce
Node 4 Z−Acceleration / 4 X−Force
ModelModel +/− 1 σ SpecificModel +/− 1 σ Generic
101
102
100
101
102
103
104
Frequency (Hz)
Acc
eler
atio
n/F
orce
Node 20 X−Acceleration / 4 X−Force
ModelModel +/− 1 σ SpecificModel +/− 1 σ Generic
101
102
100
101
102
103
104
Frequency (Hz)
Acc
eler
atio
n/F
orce
Node 4 X−Acceleration / 4 X−Force
ModelModel +/− 1 σ SpecificModel +/− 1 σ Generic
101
102
100
101
102
103
104
Frequency (Hz)
Acc
eler
atio
n/F
orce
Node 20 Z−Acceleration / 4 X−Force
ModelModel +/− 1 σ SpecificModel +/− 1 σ Generic
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-21
GENERALIZATION TO ARBITRARY APPLICATIONS
■ Response matrix
■ Singular value decomposition
( ) ( ) ( )( ) ( ) ( )
( ) ( ) ( )
=
nmmm
n
n
txtxtx
txtxtxtxtxtx
X
θθθ
θθθθθθ
;;;
;;;;;;
21
22212
12111
L
MOMM
L
L
} ( )}} ( )}
( )} ( ) ( )}
}
( )}
0
0 0
,
T
p np n pp pm m pm p T
T Tn p nm p p m p n p
T
m mm n n n
T T T Tp
T
X U V
D
D I
X X D
φη η
φ
η φ ηη φ φ
φ η
×× −×× −×
⊥ − ×− × − × −
⊥×
× ×
= Σ
=
= = =
= =
144244314442444314243
[Note: Papers use " " ]
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-22
GENERALIZATION (CONT’D)
■ Differences between analysis and test
■ Cross-orthogonality matrices
■ Normalized differences
■ Vectorization of differences
■ Structure-specific covariance matrix of uncertainty (biased)
■ Generic covariance matrix of total uncertainty (unbiased)
0
0
0
D D Dφ φ φ
η η η
∆ = −
∆ = −
∆ = −
( ) ( )00
1Trace
p
p
I
I
D D DD
ψ ψ
ν ν
∆ = −
∆ = −
∆ = −%
0
0
T
T
ψ φ φ
ν ηη
=
=
( )( )( )
∆∆∆
=∆ζvec
~vecvec
~ km
r
[ ]( )( )[ ]T
rrRR
r
rrESrE
~~~~
~
~~~
∆∆
∆
−∆−∆=
∆=
µµµ
[ ]TRR
r
rrES ~~assumed0
~~
~
∆∆==
∆Kµ
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-23
GENERALIZATION (CONT’D)
■ Consider data as function of normalized parameters
■ First order Taylor series approximation of data
■ “Propagate” uncertainty
■ Generate uncertainty bands
( ) ( ) ( )0
0, , iur ur ij
j r r
uu T r u u r u r Tr
=
∂∆ ≈ ∆ ∆ = − =
∂% %
% %
% % %%
( ) ( )vecu r X r = % %
( )( )
TUU
T Tur ur
Tur urRR
S E u u
E T r r T
T S T
= ∆ ∆
≈ ∆ ∆
≈
% %
% %% %
% %
( )
( )
110
diag
0u u
UU
U
UU n n
S
S
σ
=
O
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-24
EXAMPLE: NOSE CONE CRUSHINGNosecone aeroshell Test setup
������������
������� �� �������
��� ���������������
������ ��������
������������
Leading Edge
Impactor
DeformedShape
Aeroshell
Blivet
Simplified, axisymmetric DYNA3D model Actual buckling pattern
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-25
PREDICTIVE ACCURACY FOR NOSE CONE
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035−20
−10
0
10
20
30
40
50
Time (s)
Acc
eler
atio
n (g
)
Pre−Update Predictive Accuracy for Unit 0
Nominal Analysis
+/−σ Uncertainty Bounds
Mean Test
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035−20
−10
0
10
20
30
40
50
Time (s)
Acc
eler
atio
n (g
)
Post−Update Predictive Accuracy for Unit 0
Revised Analysis
+/−σ Uncertainty Bounds
Mean Test
Pre-update predictive accuracy Post-update predictive accuracy
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-26
EXAMPLE: BLAST RESPONSE OF R/C WALL
■ Scenario● 3-room buried bunker● Explosion in center room
■ Measurements● Displacements (12 sensors, 4 locations)● Pressures (10 sensors, 9 distinct locations)
■ Structure model● DYNA3D (customized LLNL version)● ~80,000 continuum elements● ~20,000 beam elements
■ Load model● CFD code (SHARC?)● Uncoupled from structure● Validated w.r.t. pressure measurements
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-27
GENERIC CLASS VARIATIONTypical Singular Values
01020304050607080
1 2 3 4
Mode Number
Sing
ular
Val
ue TestAnalysis
Normalized Singular Values
0
0.5
1
1.5
2
2.5
1 2 3 4
Mode Number
RM
S E
rror
12
34
12
34
0
0.2
0.4
0.6
0.8
1
RMS Error
Mode Number
Mode Number
Right Eigenvector Cross-Orthogonality
12
34
12
34
00.20.40.60.8
11.2
1.4
RMS Error
Mode Number
Mode Number
Left Eigenvector Cross-Orthogonality
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-28
PRE-UPDATE PREDICTIVE ACCURACY FOR R/C WALL
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
Left Horizontal Quarter Point Upper Vertical Quarter Point
Center Point Lower Vertical Quarter Point
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-29
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
0 0.01 0.02 0.03 0.04−1
0
1
2
3
4
5
6
7
Time (sec. full scale)
Dis
plac
emen
t (in
. ful
l sca
le)
Model Model +/− 1 σTest
POST-UPDATE PREDICTIVE ACCURACY FOR R/C WALLLeft Horizontal Quarter Point Upper Vertical Quarter Point
Center Point Lower Vertical Quarter Point
DXDynamic Experimentation Division UNCERTAINTY QUANTIFICATION WORKING GROUPO-6/28/01-30
CONCLUSIONS
■ Total uncertainty quantification requires● A generic class of test-analysis pairs● A means of normalizing the differences● A method for propagating uncertainty thru the model
■ Total uncertainty is more realistic than parametric uncertainty
■ Some unresolved questions● How are generic classes defined/interpreted?● What are the statistical issues involved?● Can this approach be made rigorous?● Can total uncertainty be used in conjunction with other types of
uncertainty quantification to deal with the issue of model uncertainty?