Post on 13-Feb-2016
description
transcript
ML: A Multilevel Preconditioning Package
Copper Mountain Conference on Iterative MethodsCopper Mountain Conference on Iterative Methods
March 29-April 2, 2004
Jonathan HuRay Tuminaro
Marzio Sala
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.
Outline
• Overview
• Multigrid basics
• Available user options
• Configuring & building
• Interoperability with other packages
• Example program
• Conclusions
ML Package
• Provides parallel multigrid preconditioning for linear solver methods• Current developers: Ray Tuminaro, Jonathan Hu, Marzio Sala
• Former: Charles Tong (LLNL)• Main methods
– Geometric• Grid refinement hierarchy• 2-level FE basis function domain decomposition
– AMG (algebraic multigrid based on aggregation)• Smoothed aggregation*• Edge-element AMG for Maxwell’s equations*• n-level (smoothed) aggregation domain decomposition *• Classical AMG
• Written primarily in C– C++ interfaces to various Trilinos packages– 102,411 lines of code (as of this afternoon)– 26 example programs– Downloadable as part of Trilinos– CVS, bugzilla
MG(f, u, k){ if (k == 1) u1 = (A1)-1f1
else { Sk(Ak, fk,uk) //pre-smooth rk = fk – Ak uk
fk-1 = Rk-1 rk; uk-1 = 0 //restrict MG(fk-1, uk-1, k-1) uk = uk + Ik-1 uk-1 //interpolate & correct Sk(Ak, fk,uk) //post-smooth
}}Ak, Rk, Ik, Sk required on all levelsSk: smoothersRk: restriction operatorsIk: interpolation operators
Recursive MG Algorithm (V Cycle)
V Cycle to solve A4u4=f4
k=4
k=1
W Cycle FMV Cycle
ML Capabilities• MG cycling: V, W, full V, full W
• Grid Transfers– Several automatic coarse grid generators– Several automatic grid transfer operators– Coarse “grid” visualization capabilities
• Smoothers– Jacobi, Gauss-Seidel, Hiptmair, Krylov methods, sparse
approximate inverses, Chebyshev
• Variety of Serial & Parallel Direct Solvers– SuperLU, Umfpack, KLU, MUMPS, etc.
• Kernels: matrix/matrix multiply, etc.
Smoothed Aggregation
• Developed for linear elasticity (Vanek, Mandel, Brezina)
• Construct tentative prolongator tP– Aggregation: group unknowns together– Augment: interpolate null space (e.g., rigid body
modes)
• Construct final prolongator P– Smooth: P = (I – D-1A)TP
• lower energy in basis functions
5
5 5 2 2
2 7
7 72
Take null space(e.g., const.) SmoothSplit into local basis
functions
Smoothed Aggregation
Uncoupled / MIS Aggregation
• Greedy algorithm
● Global graph partitioning– Operates on global domain– ParMETIS– Aggregates can span processors
Graph partitioning aggregation● Local graph partitioning
– Processors work independently– METIS– 1 aggregate per processor
ML Smoother Choices• Jacobi, Point/Block Gauss Seidel
• MLS– Based on Chebyshev polynomials of smoothed operator.– Serial: competitive with true Gauss-Seidel– Parallel: Performance is independent of # processors
• Hiptmair– Distributed relaxation smoother for Maxwell’s Eqns.– Other smoothers used within each projection step
• Aztec solvers– Krylov methods, incomplete factorizations
• Direct solution (via Amesos interface):– UMFPACK, KLU (serial)– SuperLU– MUMPS
ML and Other Packages
ML
Epetra
Accepts user data as Epetra objects
Can be wrapped as Epetra_Operator
TSF
TSF interface exists
Othermatvecs
Othersolvers
Accepts other solversand MatVecs
AmesosAmesos interface fordirect solvers
Aztecoo
Meros
Via Epetra & TSF
Configuring and Building ML
• Builds by default when you configure & build Trilinos
• By default, you get– Epetra & Aztecoo support– Example suite (Trilinos/packages/ml/examples)
• Some options of interest (off by default)– MPI support– Graph partitioning aggregation (METIS, ParMETIS)– Direct solvers (Amesos)– Profiling
configure --with-mpi-compilers=/usr/local/mpich/bin--with-ml_amesos \--with-libs=“-lamesos –lsuperlu”
A Small Example:ml/examples/ml_example_epetra_preconditioner.cpp
LinearSolver
ML: multi-grid pre-
cond.
AztecOO(Epetra)
ML(Teuchos)
gmres
AMG
Solution component
Example methods
Packagesused
Solve Ax=b:A: advection/diffusion operatorLinear solver: gmresPrecond.: 2-level AMG, graph-
partitioning aggregation
ml_example_epetra_preconditioner.cpp
Trilinos_Util_CrsMatrixGallery Gallery(“recirc_2d", Comm); Gallery.Set("problem_size", 10000); // linear system matrix & linear problem Epetra_RowMatrix * A = Gallery.GetMatrix(); Epetra_LinearProblem * Problem = Gallery.GetLinearProblem();
// Construct outer solver object AztecOO solver(*Problem);
// Set some solver options solver.SetAztecOption(AZ_solver, AZ_gmres); solver.SetAztecOption(AZ_output, 10); solver.SetAztecOption(AZ_kspace, 160);
example (contd.) // Set up multilevel preconditioner ParameterList MLList; // parameter list for ML options
MLList.set("max levels",2); MLList.set("aggregation: type", "METIS"); // graph partitioning MLList.set("aggregation: nodes per aggregate", 16);
// set up aztecoo smoother MLList.set("smoother: type","aztec"); int options[AZ_OPTIONS_SIZE]; double params[AZ_PARAMS_SIZE]; AZ_defaults(options,params); options[AZ_precond] = AZ_dom_decomp; options[AZ_subdomain_solve] = AZ_ilut; MLList.set("smoother: aztec options", options); MLList.set("smoother: aztec params", params);
MLList.set("coarse: type","Amesos_Superludist"); MLList.set("coarse: max processes", 4);
ML_Epetra::MultiLevelPreconditioner * MLPrec = new // create preconditioner ML_Epetra::MultiLevelPreconditioner(*A, MLList, true);
solver.SetPrecOperator(MLPrec); // tell solver to use ML preconditioner solver.Iterate(500, 1e-12); // iterate at most 500 times
AMG for Common Problem Types Trilinos_Util_CrsMatrixGallery Gallery(“laplace_3d", Comm); Gallery.Set("problem_size", 100*100*100); // linear system matrix & linear problem Epetra_RowMatrix * A = Gallery.GetMatrix(); Epetra_LinearProblem * Problem = Gallery.GetLinearProblem();
// Construct outer solver object AztecOO solver(*Problem); solver.SetAztecOption(AZ_solver, AZ_cg);
// Set up multilevel precond. with smoothed aggr. defaults ParameterList MLList; // parameter list for ML options ML_Epetra::SetDefaults(“SA”,MLList); ML_Epetra::MultiLevelPreconditioner * MLPrec = new // create preconditioner ML_Epetra::MultiLevelPreconditioner(*A, MLList, true);
solver.SetPrecOperator(MLPrec); // tell solver to use ML preconditioner solver.Iterate(500, 1e-12); // iterate at most 500 times
Other problem types: “DD”, “DD-ML”, “maxwell”
Collaborations• Within Sandia
– ALEGRA• Radiation• Maxwell• Electrostatic potential
– MPSalsa (Shadid, et al.)– CEPTRE
• External users– EM3D (Lawrence Berkeley NL)– P. Arbenz (ETH Zurich)– R. Geus (PSI)– J. Fish, H. Waisman (RPI)
• Potential Users– PREMO (Sandia)
Getting Help
• Website: www.cs.sandia.gov/~tuminaro/ml• See Trilinos/packages/ml/examples• See guide:
– ML User’s Guide, ver. 3.0 (in ml/doc)• Mailing lists
– ml-users@software.sandia.gov– ml-announce@software.sandia.gov
• Bug reporting, enhancement requests via bugzilla:– http://software.sandia.gov/bugzilla
• Email us directly– jhu@sandia.gov– rstumin@sandia.gov– msala@sandia.gov
Conclusions
• ML 3.0 release this May– Part of next Trilinos release
• Existing options– Uncoupled / MIS aggregation– Smoothers– SuperLU
• New options with ML 3.0– Graph-based aggregation– “parameter list” setup– Amesos interface to direct solvers– Visualization, aggregate statistics
• Use ML with Trilinos!– ML as preconditioner is trivial– Rich set of external libraries