High-Performance Computing (HPC)IS
Transforming Seismology
Southern San Andreas Earthquake• M 7.7, scaled Denali slip• SCEC CVM3 (600 km x 300 km x 80 km)• 3000 x 1500 x 400 = 1.8 G nodes (200 m)• 20,000 time steps (0.01 s)• 19,000 SU per run• 47 TB of simulation data (150,000 files) per run
TeraShake 1 (Olsen et al. 2006) 1012 flops
Energy Funneling Effect (Olsen et al., 2006)
Blue: data
Red: synthetic
16 Jun 2005, ML4.9, Yucaipa earthquake
Data
Synthetic
Reference model: SCEC Community Velocity Model 3.0
HPC makes seismic wave propagation simulations more realistic and more accurate, opens up the possibility for physics-based, deterministic, seismic hazard analysis.
Let’s watch a video made by SCEC.
Two Problem Areas1. Develop simulation capability for physics-based seismic hazard and risk analysis
- TeraShake platform- CyberShake project
2. Improve physical models for SHA- Inversion of large data sets for Unified Structural Representation
SCEC computational pathways
AWM: Anelastic Wave ModelFSM: Fault-system ModelRDM: Rupture Dynamics ModelSRM: Site-response Model
Realistic 3D Earth Structure Model (CVM)
+ High-Performance Computing
(HPC)=
CyberShake
€
G(x i;x r )
€
G(x r;x i) = [G(x i;x r)]T
Receiver Green Tensor (RGT)• Obtain Green tensors from a receiver to all grid points by finite difference simulations (3 runs for 3 orthogonal forces at receiver).
• Reciprocity states that the Green tensors from all the grid points to the receiver is the transpose of the RGT obtained above.
• Synthetic seismograms due to an arbitrary point source s at receiver r and their gradients with respect to source locations can be retrieved from the RGT database.
3D Earth Structural Model
rS(l-1, m, n)
(l, m, n)(l+1, m, n)
h(l, m+1, n)
(l, m-1, n)
(l, m, n-1)
(l, m, n+1)
Red dash line: synthetics from RGT and reciprocity
Blue solid line: synthetics from forward wave propagation
Confirm Reciprocity
Numerical differentiation to get receiver strain Green tensor
Yorba Linda Earthquake to basin station BRE
Physics-based Seismic Hazard Analysis
(CyberShake)
€
u(x r, t) = dV (x s) dts∫ ∫ M(x s,ts) :∇ sGT (x s,t − ts;x r)€
∇sGT (x s,t − ts;x r )
€
M(x s,ts)
€
u(x r, t)
Callaghan et al. (2006)
Red: empirical ground motion model (Abrahamson & Silva 1997)
Black: CyberShake (Callaghan 2006)
Two Problem Areas1. Develop simulation capability for physics-based seismic hazard and risk analysis
- TeraShake platform- CyberShake project
2. Improve physical models for SHA- Inversion of large data sets for Unified Structural Representation
SCEC computational pathways
AWM: Anelastic Wave ModelFSM: Fault-system ModelRDM: Rupture Dynamics ModelSRM: Site-response Model
1 2 3 4 5 6 7 8Magnitude
Centroid Moment Tensor (CMT)
Finite Moment Tensor (FMT)
Isotropic Point
Source (IPS)
FaultSlip
Distribution (FSD)
(5) (8-10) (13-20) (>100)
Number of parameters
Seismic Source Parameter Inversion
€
u(x r, t) = dV (x s) dts∫ ∫ M(x s, ts) :∇ sGT (x s, t − ts;x r)
Numerical tests to verify inversion algorithm
Waveform inversion using 3D RGT synthetics .vs. first-motion focal mechanisms
Rapid CMT Inversion Using Waveforms computed in a 3D Earth Structural Model
Yorba Linda Cluster
Fontana Trend
A new left-lateral fault?
Resolving Fault-plane-ambiguity for Small Earthquakes
A new representation of finite moment tensor
Fréchet Kernel for Full-wave Tomography
Born Approximation:
Born Kernels
Seismogram perturbation kernel:
Data functional:
Fréchet kernel:
Receiver Green Tensor
δtp= -0.4 (s)
∫⊕
= rrr dKt mpp )()( δαδ
LAB Inversion Computational Cost For One GN Iteration
F3DT for Southern California (TERA3D)
• Target frequency: 1.0 Hz for body-waves and 0.5 Hz for surface waves
• Starting model: 3D SCEC CVM4
• Grid-spacing 200m, spatial grid points 1871M
• 150 stations, 200 earthquakes, 650 simulations, 5.2M CPU-Hrs
• Octree-based data compression, 895TB storage