Sandia National Laboratories is a multimis- sion laboratory managed and operated by National Technology and Engineering So- lutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525. SAND NO. 2018-4438 C Utilizing Distributional Measurements of Material Characteristics from SEM Images for Inverse Prediction PRESENTED BY Daniel Ries 1 Contributors: John R. Lewis 1 , Adah Zhang 1 , Christine M. Anderson-Cook 2 , Marianne Wilkerson 2 , Gregory L. Wagner 2 , Julie Gravelle 2 , Jacquelyn Dorhout 2 1 Sandia National Laboratories 2 Los Alamos National Laboratory
Transcript
Sandia National Laboratories is a multimis-sion laboratory managed and operated byNational Technology and Engineering So-lutions of Sandia, LLC., a wholly owned
subsidiary of Honeywell International, Inc.,for the U.S. Department of Energy’s National
Nuclear Security Administration under contractDE-NA-0003525. SAND NO. 2018-4438 C
Utilizing Distributional Measurements of MaterialCharacteristics from SEM Images for Inverse Prediction
PRESENTED BY
Daniel Ries1
Contributors: John R. Lewis1, Adah Zhang1,Christine M. Anderson-Cook2,Marianne Wilkerson2, Gregory L. Wagner2,Julie Gravelle2, Jacquelyn Dorhout2
1Sandia National Laboratories 2Los Alamos National Laboratory
2 Introduction
Experiments are being conducted at US National Labs in nuclearforensics with the goal of exploring the impact of differentproduction and processing parameters on materials produced.
Underlying Goal: Build a model from which interdicted materialscan be matched to their original production environments usingmorphology information from SEM images of the interdictedmaterial.
⇒ This approach is referred to as inverse prediction because it’sgoing in opposite direction of causality.
• 2 areas on slide examined at 5000x, 10000x, 15000x, 25000x• 8 total SEM images per run
May 2, 2018
4 Sample SEM Image With Segmentation inMAMA Software
May 2, 2018
5 Bench-Scale Uranium SEM DataUsing MAMA (Morphological Analysis of MAterials) software, thefollowing are measured for each particle in each SEM image:
However, a single sample has multiple particles⇒ multiplemeasurements of same characteristic for one set of experimentalconditions!
This allows us to consider distributional responses instead ofsingle number summaries.
Standard Approach (Aggregation): For each experimental run,take the average over all measurements for each responsevariable.
Our Approach: Estimate cumulative distribution functions (cdf)for each response of each experimental run.
May 2, 2018
7 Average Response For Select Responses andInputs
May 2, 2018
8 Cumulative Distribution FunctionDefinition: Cumulative Distribution Function (CDF)A CDF is a function of x that returns the probability of being lessthan or equal to x
10 12 14 16 18 20
0.0
00
.05
0.1
00
.15
0.2
00
.25
0.3
0
PDF
Response
Fre
qu
en
cy
0.8
10 12 14 16 18 20
0.0
0.2
0.4
0.6
0.8
1.0
CDF
Response
P(X
< R
esp
on
se)
P(X < 16) = 0.8
May 2, 2018
9 Bench-Scale Distributional Responses
0
5
10
15
20
0.8 0.9
Perimeter Convexity
Freq
uenc
y StirRate
170
280
400
PDF
0.00
0.25
0.50
0.75
1.00
0.8 0.9
Perimeter Convexity
CD
F
StirRate
170
280
400
CDF
May 2, 2018
10 Understanding Performance Via SimulationStudySimulated X: 100 different values of XSimulated Y: For each X, a distribution Y values are sampled (size100)
−2
−1
0
1
0.0 2.5 5.0 7.5 10.0
X
Mea
n Y
• Mean of Y is constant for all values of X
• As X increases, variance of the response increasesMay 2, 2018
11 Understanding Performance Via SimulationStudy
0.00
0.25
0.50
0.75
1.00
−10 −5 0 5 10
Values of Y
CD
F o
f Y
2.5
5.0
7.5
x
May 2, 2018
12 Understanding Performance Via SimulationStudy
n=50 n=100
−0.5 0.0 0.5 −0.5 0.0 0.50.1
0.2
0.3
0.4
0.5
0.6
ρ
PM
SE
variable
q=1
q=2
N
50
100
PMSE: Prediction Mean Squared Error (smaller means less left unexplained)n: number of experimental runsN: observations per experimental runq: number of response variables
PMSE for standard method: 18.1!May 2, 2018
13 Bench-Scale Uranium Distributions of SelectResponses
0.00
0.25
0.50
0.75
1.00
0.8 0.9
Perimeter Convexity
CD
F
StirRate
170
280
400 0.00
0.25
0.50
0.75
1.00
0.8 0.9
Perimeter Convexity
CD
F
FlowRate
2.5
5
7.5
0.00
0.25
0.50
0.75
1.00
−7.5 −5.0 −2.5 0.0 2.5 5.0
log Vector Area
CD
F
StirRate
170
280
400 0.00
0.25
0.50
0.75
1.00
−7.5−5.0−2.5 0.0 2.5 5.0
log Vector AreaC
DF
FlowRate
2.5
5
7.5
0.00
0.25
0.50
0.75
1.00
1 2 3
Ellipse Aspect Ratio
CD
F
StirRate
170
280
400 0.00
0.25
0.50
0.75
1.00
1 2 3
Ellipse Aspect Ratio
CD
F
FlowRate
2.5
5
7.5
May 2, 2018
14 Inverse Prediction on Bench-Scale UraniumData
UNO3ratio StirRate FlowRate EndpH TempStandard-5 Y 83.31 135.70 3.67 3.72 19.74Functional-5 Y 79.35 81.53 3.42 2.65 16.89Functional-3 Y 84.84 84.28 3.33 2.71 16.64Functional-1 Y 76.99 85.30 3.37 2.79 16.41
Table: Root PMSE using original scale data.
• Standard-5 Y: Standard method using vector area, ellipse aspectratio, perimeter convexity, ecd, area convexity
• Functional-5 Y: Functional method using vector area, ellipse aspectratio, perimeter convexity, ecd, area convexity
• Functional-3 Y: Functional method using only vector area, ellipseaspect ratio, and perimeter convexity
• Functional-1 Y: Functional method using only vector areaMay 2, 2018
15 Conclusions
We presented a method that utilizes the SEM morphologydistributional responses directly to perform inverse prediction
• Simulation study and real data results show improvementsover the current standard method.
• Simulation study suggests that we only need to analyze ≈50particles per run to estimate the cdf well.
• Real data results are only based on a small 18-runexperiment.
• We expect significant improvements in predictive capabilityas the number of experimental runs increases, as evidencedby simulation study.
