Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Eric Phippsand
Andy SalingerApplied Computational Methods Department
Sandia National Laboratories
9th Copper Mountain Conference on Iterative MethodsApril 7, 2006
Solving Bordered Systems of Linear Equations for Large-Scale
Continuation and Bifurcation Analysis
Why Do We Need Stability Analysis Algorithms for Large-Scale Applications?
Nonlinear systems exhibit instabilities, e.g.:
• Buckling• Ignition • Onset of Oscillations• Phase Transitions
These phenomena must be understood in order to perform computational design and optimization.
Established stability/bifurcation analysis libraries exist:
• AUTO (Doedel, et al)• CONTENT (Kuznetsov, et al)• MATCONT (Govaerts, et al)• etc…
We need algorithms, software, and experience to impact ASC- and SciDAC-sized applications (millions of unknowns)
Stability/bifurcation analysis provides qualitative information about time evolution of nonlinear systems by computing families of steady-state solutions.
LOCA: Library of Continuation Algorithms
LOCA: Library of Continuation Algorithms
LOCA provides:• Parameter Continuation: Tracks a family of
steady state solutions with parameter
• Linear Stability Analysis: Calculates leading eigenvalues via Anasazi (Thornquist, Lehoucq)
• Bifurcation Tracking: Locates neutral stability point (x,p) and tracks as a function of a second parameter
Application code provides:• Nonlinear steady-state residual and Jacobian fill:
• Newton-like linear solves:
External force
Second parameter
Ext
erna
l for
ce1
1
3
LOCA Designed for Easy Linking to Existing Newton-based Applications
Algorithmic choices for LOCA: • Must work with iterative (approximate) linear solvers on
distributed memory machines• Non-Invasive Implementation (e.g. matrix blind) • Should avoid or limit:
Requiring more derivatives Changing sparsity pattern of matrix Increasing memory requirements
LOCA targets existing codes that are:• Steady-State, Nonlinear• Newton’s Method• Large-Scale, Parallel
Bordering Algorithms Meet These Requirements
… but 4 solves of J per Newton iteration are used to drive J singular!
Turning Point Bifurcation Full Newton Algorithm
Bordering Algorithm
These Techniques Have Been Applied To Many Interesting Problems
-2.6
-2.4
-2.2
-2
-1.8
-1.6
-1.4
0 0.1 0.2 0.3 0.4 0.5
mixedperp
free energy
εwA
Capillary Condensation Flow in CVD Reactor Yeast Cell-Cycle Control
Buckling of Garden Hose Block Copolymer Self-Assembly Propane&Propylene Combustion
LOCA has been rewritten as part of the Trilinos framework
Better algorithms can be implemented with tighter coupling to linear algebra
LOCA can build upon the existing interface between application codes and the NOX nonlinear solver
Sandia’s parallel iterative solvers are made available to LOCA users
• Collection of parallel solver packages, with a common build process, made interoperable where appropriate (Heroux et al)
Trilinos 6.0 Released: September 1, 2005
2004
2004HPC Software Challenge
Bordered Systems of Equations
Only requires solves of J but • Requires m+l linear solves
• Has difficulty when J is nearly singular
Solving bordered systems of equations is a ubiquitous computation:
Pseudo-Arclength Continuation
Constraint Following
Turning Point Identification
Bordering Algorithm
Solving Bordered Systems via QR
1H.F Walker, SIAM J. Sci. Comput., 19992R. Schreiber, SIAM J. Stat. Comput., 1989
Extension of Householder pseudo-arclength technique by Homer Walker1
QR Factorization Compact WY Representation2
where
Rearranged Bordered System
Write then
P is nxn, nonsingular, rank m update to J
Snap-through Buckling of a Shallow Cap(66K tri-shell elements, 200K unknowns, 16 procs)
Salinas (Reese et al., SNL):
• Unstructured finite element, linear elasticity
• Corotational formulation for beams and shells (C. Felippa, CU-Boulder)
• Updated Lagrangian for solid elements
• Analytic, Sparse Jacobian
• Fully Coupled Newton Method (NOX)
• GMRES (Aztec) with RILU(k) Preconditioner (Ifpack)
• Distributed Memory Parallelism
QuickTime™ and aYUV420 codec decompressor
are needed to see this picture.
Turning Point IdentificationMoore-Spence Formulation
… but 4 solves of J per Newton iteration are used to drive J singular!
Turning Point Bifurcation Full Newton Algorithm
Bordering Algorithm
Snap-through Buckling of a Shallow CapTurning Point Bordering Method
• RILU fill factor: 6
• RILU overlap: 6
• Krylov space: 2000
•
Solve 5 bordered systems of equations using QR approach
Then
Modified Turning Point Bordering Algorithm
Snap-through Buckling of a Shallow CapModified Turning Point Bordering Method
• RILU fill factor: 6
• RILU overlap: 6
• Krylov space: 2000
•
Given and , let
then
There are constants such that
Standard formulation:
Note for Newton’s method:
3 linear solves per Newton iteration (5 for modified bordering)!For symmetric problems reduces to 2 solves.
Minimally Augmented Turning Point Formulation
With iterative solvers,
so define
where
Also
Minimally Augmented Turning Point Formulation for Large-Scale Problems
Snap-through Buckling of a Shallow CapMinimally Augmented Formulation
• RILU fill factor: 6
• RILU overlap: 6
• Krylov space: 2000
• Modified bordering
• Minimally augmented
Pseudo-Arclength Continuation of Turning Points
Pseudo-Arclength Equations
Minimally Augmented: 3 total linear solves
Moore-Spence w/Bordering:7 total linear solves
Newton Solve
Snap-through Buckling of a Symmetric Cap Pseudo-arclength Turning Point Continuation
Method Continuation Steps
Failed Steps
Nonlinear Iterations
Linear Solves
Linear Iterations
Total Time (hrs)
Moore-Spence Mod. Bordering
13 1 54 427 117797 4.2
Min. Augmented 12 0 49 122 21888 1.2
Summary
• QR approach provides a convenient way to solve bordered systems– Nonsingular– Only involves one linear solve– Only requires simple vector operations– Doesn’t change dimension of the linear system– Become a workhorse tool in LOCA
• Highly encouraged by minimally augmented turning point formulation– No singular matrix solves– Improves robustness, scalability and accuracy– Requires linear algebra-specific implementation
• Future work– Minimally augmented pitchfork, Hopf bifurcations– Preconditioners for
Points of Contact• LOCA: Trilinos continuation and bifurcation package
– Sub-package of NOX– Andy Salinger ([email protected])– Eric Phipps ([email protected])– www.software.sandia.gov/nox
• NOX: Trilinos nonlinear solver package– Roger Pawlowski ([email protected])– Tammy Kolda ([email protected])– www.software.sandia.gov/nox
• Trilinos: Collection of large-scale linear/nonlinear solvers– Epetra, Ifpack, AztecOO, ML, NOX, LOCA, …– Mike Heroux ([email protected])– www.software.sandia.gov/Trilinos
• Trilinos Release 6.0 currently available, 7.0 this fall– 7.0 will include all LOCA algorithms presented here
• Salinas: Massively Parallel Structural Dynamics Finite Element Code– Garth Reese ([email protected])
With iterative solvers, solves for u and v are in-exact. Does this impact nonlinear convergence? Instead solve for updates to u and v every nonlinear iteration:
Modified Minimally Augmented Turning Point Formulation for Large-Scale Problems
Snap-through Buckling of a Shallow CapModified Minimally Augmented Formulation
• RILU fill factor: 6
• RILU overlap: 6
• Krylov space: 2000
•