Post on 26-Feb-2016
description
transcript
Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear
Security Administration under contract DE-AC04-94AL85000.
First-Order Stokes Model
With Glen’s Law Viscosity
Available BCs:No-Slip Basal SlidingStress-Free Open Ocean
Objectives
A.G. Salinger, I. Kalashnikova, M. Perego, R.S. Tuminaro, M.S. Eldred and J.D. Jakeman, Sandia National LaboratoriesS. Price, M. Hoffman, Los Alamos National Laboratories
Ongoing Work
Development of the Albany/FELIX Land Ice Dycore using Software Components
SAND 2014-xxxxP
Component-Based StrategyComponent-based approach enables rapid development of new production codes embedded with transformational capabilities
Element Level FillMaterial Models
Sensitivities
Field ManagerDiscretization Library
Remeshing
UQ Solver
Nonlinear SolverTime Integration
Optimization
Objective Function
Local Fill
Mesh Database
Mesh ToolsI/O Management
Input File ParserUtilities
UQ (sampling)Parameter Studies
Mesh I/O
Optimization
Geometry Database
Discretizations
Derivative Tools
AdjointsUQ / PCE
Propagation
ConstraintsError Estimates
Continuation
Constrained Solves
Sensitivity AnalysisStability Analysis
V&V, CalibrationParameter List
VerificationVisualization
PostProcessing
AdaptivityModel Reduction
Memory ManagementSystem Models
MultiPhysics Coupling
OUU, Reliability
Communicators
PartitioningLoad Balancing
Analysis Tools (black-box)
Physics Fill
Composite Physics
Data Structures
Direct Solvers
Linear Algebra
Architecture-Dependent Kernels
Preconditioners
Iterative Solvers
Eigen Solver
System UQ
Analysis Tools (embedded)
Matrix Partitioning
Inline Meshing
MMS Source Terms
Grid TransfersQuality Improvement
Mesh Database
Solution Database
Derivatives
Regression Testing
Bug Tracking
Version Control
Software Quality
Porting
Performance TestingCode Coverage
Mailing Lists
Release Process
Unit Testing
Web Pages
Build SystemBackups
Verification Tests
DOF map
Multi-CoreAccelerators
Linear Programming
Graph AlgorithmsData-Centric Algs
SVDsMap-Reduce
Network Models
Dycore Interfaces and Meshes
UQ: Bayesian Calibration
Convergence & Scalability
Sandia’s components effort includes ~100 interoperable libraries
Solution Verificationusing manufactured solutions
Defining a UQ workflow for stochastic inversion of Basal sliding coefficients: 1. Model Reduction (KLE) 2. PCE Emulator 3. MCMC Calibration using Emulator
Albany/FELIX Ice Sheet Dycore
Develop: robust and scalable unstructured-grid finite element ice sheet code: Stand-alone steady-state model for initialization and calibration Dynamic model when linked to MPAS-LI or CISM for advection Future land ice component of DOE-ACME earth system model
Support: DOE climate missions, such as providing Sea Level Rise predictionsLeverage: software and expertise from SciDAC Institutes (FASTMath, QUEST, SUPER) and hardware from DOE Leadership Class Facilities
Funding: “PISCEES” SciDAC Application Partnership (DOE’s BER + ASCR divisions)PIs: S. Price and E. Ng; collaboration with ORNL, LANL, LBNL, UT, FSU, SC, MIT, and NCAR
Mature dynamic evolution capability under MPAS Perform deterministic and stochastic initialization runs Improve coupling to full earth system model Finish conversion to performance-portable kernelsWe acknowledge the contributions of our PISCEES collaborators, including B. Lipscomb, K. Evans, P. Worley, M. Norman, M. Gunzberger, and C. Jackson, and our many Trilinos/Dakota collaborators, including E. Phipps and L. Swiler
• Finite Element Discretization (Hex, Tet)• Parallel, Unstructured Grid with Partitioning• Automatic Differentiation for Jacobians• Globalized Newton’s Method Nonlinear Solves• Preconditioned Krylov Iterative Solvers• Performance-Portable Kernels (in progress)• Software tools: git / cmake / ctest / jenkins
g=10-1.0
g=10-2.5
g=10-6.0
g=10-10
g=10-10
4 cores334K DOFs
(8km GIS, 5 layers)
16384 cores1.12B DOFs
(0.5km GIS, 80 layers)
Robust Nonlinear Solvesusing Homotopy Continuation
3D Mesh convergence study for GIS model gives theoretical 2nd-Order rate
How many vertical layers do you need? Convergence study for GIS 1km mesh:
Scalability results over 4 mesh bisections:
84
The Albany/FELIX solver can be driven by a CISM or MPAS-LI interface:
CISM MPAS
CISM MPAS-LI
T=70 yrT= 0 yr
We are beginning to do dynamic runs:
CISM (Fortran)Thickness evolution, temperature solve,
coupling to ESM
simple_glide Albany/FELIX (C++)velocity solve
MPAS/Land Ice (Fortran)Thickness evlolution,
temperature solve,coupling to ESM
LandIce_model
C++/Fortran interface,
mesh conversion
C++/Fortran interface,
mesh conversion
• Structured rectangles• Extruded to Hexs
• Unstructured polygons• Dual mesh of triangles• Extruded to Tets
Regional Refinement:
# vertical layers/# cores
# dofs Total Time -
Setup (sec)
Solution Average
Error
5/128 21.0M 519.4 2.827 3.17e-2
10/256 38.5M 525.4 2.896 8.04e-3
20/512 73.5M 499.8 2.924 2.01e-3
40/1024 143M 1282 2.937 4.96e-4
80/2048 283M 1294 2.943 1.20e-4
160/4096 563M 1727 2.945 2.76e-5