1 1 What does Performance Across the Software Stack mean? High level view: Providing performance for physics simulations meaningful to applications To FASTMath it means • Providing performance for people solving large systems of equations • Providing performance for mesh-based simulations • Providing performance for multiphysics simulations which can include a combination of mesh and particle based methods Will focus on performance for mesh-based simulations Performance Across the Software Stack
Transcript
1
1
What does Performance Across the Software Stack mean? High level view: Providing performance for physics
simulations meaningful to applications To FASTMath it means• Providing performance for people solving large systems of
equations• Providing performance for mesh-based simulations• Providing performance for multiphysics simulations which
can include a combination of mesh and particle based methods
Will focus on performance for mesh-based simulations• Even at this level there is a substantial difference between
structured and unstructured mesh methods
Performance Across the Software Stack
2
2
(A simplified view by an unstructured mesh person – need proper technical description from structured mesh people)
Core components of adaptive structured mesh system Defining the physics on the grids Management of the grids and their interaction Dynamic load balancing of the grids Assembly of algebraic equations Solving of the systems of algebraic equations over the grids
Implementation approach used Provide an integrated infrastructure for the grids and the
solutions of the systems of algebraic equations over the grids Provide hooks for adding physics through defining the input
to stencil operations
Structured Mesh Simulations
3
3
Software implementations of structured mesh methods Provide a parallel framework to support• The interactions of the grids• Adaptive control of the grids• Assemble of the systems of algebraic systems• Solution of the algebraic systems• Defining stencil operations to account for physics
Key software stack integration points• Integration with solver libraries (in reality influenced by
both the grid infrastructure and physics being solved)• Integration of new physics (note – expect it is not as
simple as it “provide a new stencil sounds”)
Structured Mesh Simulations
4
4
Historic focus has been on the Physics equation discretization alternatives Construction and solution of the algebraic systems Result is development of unstructured mesh analysis codes
Fact that unstructured mesh generation/adaptation can be automated introduced the need for a mesh infrastructure
Two approaches to supporting unstructured mesh workflows Components that can integrate with existing unstructured
mesh simulation codes Provide an unstructured mesh simulation infrastructure for
solving new problems SciDAC applications require both options
Unstructured Mesh Simulations
5
5
Mesh generation and adaptation can be automated using design data input• Any combinations of CAD and triangulations• Voxel (image) to model to mesh capabilities• Extensive control of mesh types,
orders and layouts – boundary layer, anisotropic, gradation, etc.• Curved element meshes• Parallel mesh and distributed geometry• Mesh generation in parallel• Mesh adaptation in parallel
Mesh Generation and Adaptation
6
6
Need uniform interface to solve multi-component problems on unstructured meshes Create a native MOAB implementation that exposes the
underlying array-based mesh data structures through the DM (Discretization Manager) object in PETSc (DMMoab)
Discretize the physics PDE described on MOAB mesh while leveraging the scalability of PETSc solvers.
Provide routines to build simple meshes in-memory or load an unstructured grid from file.
Analyze efficient unstructured mesh traversal, FD/FEM-type operator assembly for relevant problems in multi-dimensions.
(includes linkage to geometry)• Fields• Solution transfer
Dynamic load balancing Unstructured mesh solver• Physics equation discretization• System formation• System solution
Component Based Approach
iM0
jM1
1P
0P2P
inter-process part boundary
intra-process part boundary
Proc j Proc i
9
9
Parallel Data & Services
Domain Topology
Mesh Topology/Shape
Dynamic Load Balancing
Simulation Fields
Physics and Model Parameters Input Domain Definition with Attributes
Mesh-Based Analysis
Complete Domain
Definition
Mesh Generation and/or Adaptation
Postprocessing/Visualization
SolutionTransfer
Correction Indicator
PDE’s anddiscretizationmethods
Solution transfer constraints
mesh with fields
mesh with fields
calculated fields
mesh size field
meshes and fields
meshing operation geometric
interrogation
Attributed topology
non-manifoldmodel construction
geometry updates
mesh size field
mesh
Partition Control
Components in Parallel Adaptive Analysis
10
10
File transfer a serious bottleneck in parallel simulation workflows All core parallel data and services accessed through APIs In-memory integration approach uses APIs• Migration from file-based components to in-memory
Modify/extend components by wrapping data structures with APIs for: read/write memory management inter-language coupling; typically FORTRAN and C
In-memory has far superior parallel performance
In-Memory Coupling of Simulation Components
11
11
Electromagnetics Analysis
12
12
Adaptation based on• Tracking particles • Discretization errors
Full accelerator models• Approaching 100 cavities• Substantial internal structure• Meshes with several
hundred million high-order curved elements
High-Order EM Coupled with PIC
13
13
Structural Analysis for Integrated Circuits
14
14
Must construct 3-D non-manifold solid from input geometry Input domain defined in terms of 2-D layouts (gdsII/OASIS) Third dimension based on process knowledge A component has been developed to construct the model
Adaptive loop constructed for thermally loaded case including thin liner
