Algebraic Multigrid as Solvers and as Preconditionerta.twi.tudelft.nl/.../week1_part2_slides.pdf ·...

Algebraic Multigrid as Solvers and as Preconditioner

Domenico [email protected]

http://www.cs.kuleuven.ac.be/∼domenico/

Department of Computer ScienceKatholieke Universiteit Leuven

Celestijnenlaan 200 AB-3001 Heverlee

Belgium

n

Structure of the Presentation

⊲⊳ Motivation for Iterative solvers

⊲⊳ Stationary Iterative Solvers

⊲⊳ Multigrid Methods

⊲⊳ Algebraic Multigrid Methods

⊲⊳ Krylov Subspace Methods

⊲⊳ Algebraic Multigrid Methods as Preconditioner

M Structure of the Presentation Domenico Lahaye

Motivation for Iterative solvers

⊲⊳ 2D Eliptic partial differential equation − ∂

∂x

(ν

∂u

∂x

)− ∂

∂y

(ν

∂u

∂y

)= f

⊲⊳ Discretization by e.g. finite difference or finite elements

⊲⊳ Resulting system of linear equation A x = b

∗ A discrete differential operator∗ x and b unknown function u and f in discrete points respectively

⊲⊳ dim(A) large (acurate discretizations)

⊲⊳ A x = b has to be solved repeatedly (parameter studies)

⇒ Fast and memory efficient solution procedures required

M Motivation for Iterative solvers Domenico Lahaye

Motivation for Iterative solvers (2)

Example: Numerical models for electrical motors

⊲⊳ A = (0,0, u) B = ∇× A

H = ν B

⊲⊳ ⇒ computation of forces and torques

⊲⊳ A x = b dim(A) ∼ 105 − 106

⊲⊳ solved in time stepping oroptimization loop

M Motivation for Iterative solvers (2) Domenico Lahaye


Iterative methods

Given x0, compute xmm≥1 such that‖ em ‖=‖ x − xm ‖→ 0 if m → ∞

⊲⊳ stationary methods ek = Mke0∗ multigrid methods∗ domain decomposition methods

⊲⊳ non-stationary methods∗ Krylov subspace methods

Matrix-vector multiply as computational kernel⇒ sparsity exploited⇒ A v in O(N) flops, N = dim(A)

Direct methods

Rely on a factorization of A



Representative sparsity structure of A

0 20 40 60 80 100 120 140 160 180

0

20

40

60

80

100

120

140

160

180

1203








⊲⊳ Algebraic Multigrid Methods as Precondi-tioner

⊲⊳ Conclusions


Basic Iterative Schemes for A x = b

⊲⊳ Matrix splitting A = M − N with M−1 v “easy” to compute

⊲⊳ Iterative scheme xm+1 = M−1 N xm + M−1b (iterand)em+1 = (I − M−1 A) em (error)

⊲⊳ A = D − L − U ∗ Jacobi method MJAC = D (diagonal)∗ Gauss-Seidel method MGS = D − L (triangular)∗ Successive relaxation MSOR = 1

ωD − L ω > 0

⊲⊳ Asymptotic rate of convergence given by ρ(I − M−1 A) or ‖ I − M−1 A ‖

M Basic Iterative Schemes for A x = b Domenico Lahaye

Basic Iterative Schemes for A x = b (2)

Model Problem Analysis

⊲⊳ Continuous problem: − u = f on Ω = [0,1]× [0,1] + boundary conditions

⊲⊳ Discretization on uniform mesh, mesh width h, using central finite differences

⊲⊳ Linear system A x = b with [A] = 1h2

0 −1 0−1 4 −10 −1 0

Ωh =

M Basic Iterative Schemes for A x = b (2) Domenico Lahaye


Convergence history of lexicografic Gauss-Seidel (fixed mesh)

∗ for a fixed mesh (fixed h )initial stage: fast convergencelater stage: stalling convergence

0 10 20 30 40 50 6010

1

102

103

104

Number of iterations

Res

idua

l



Convergence history of lexicografic Gauss-Seidel (varrying mesh size)

∗ for finer meshes ( h → 0 )‖ I−M−1 A ‖= 1−O(h2)

⇒ increasing number ofiterations

0 10 20 30 40 50 6010

−2

10−1

100

101

102

103

104

105

h = 1/8

h = 1/16

Number of iterations

Res

idua

l h = 1/32



Slow convergence is caused by smooth error components

Initial error After 5 iterations After 10 iterations

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.1

−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.05

0

0.05

0.1

0.15

0.2

0.25

Smooth error components can be represented on coarser grids









⊲⊳ Conclusions


Multigrid Methods

⊲⊳ exploit PDE background of the linear problem⊲⊳ multigrid methods = smoother (basic iterative scheme) + coarse grid correction⊲⊳ two-grid scheme

applypre

smoothing

projecton coarser

grid

correction

addcorrection

apply

smoothingpost

coarse gridcompute

fine gridto

interpolate

xhk xh

k+1

⊲⊳ multigrid scheme by recursive application to solve coarse grid problem

M Multigrid Methods Domenico Lahaye

Multigrid Methods (2)

Geometric multigrid methods

⊲⊳ Ah xh = bh on Ωh = ⇒ AH xH = bH on ΩH =

⊲⊳ construct restriction IHh : Ωh → ΩH interpolation Ih

H : ΩH → Ωh

⊲⊳ coarse grid correction Kh,H = Ih − IhH (AH)−1 IH

h Ah

smoother Sh = (I − (Qh)−1 Ah) Qh splitting of Ah

multigrid iteration matrix Mh,H(ν1, ν2) = (Sh2)

ν2 Kh,H (Sh1)

ν1

M Multigrid Methods (2) Domenico Lahaye


Geometric multigrid methods (2)

⊲⊳ Different orders in visiting coarser grids ⇒ different cycles (V-, W-, F-cycle)

⊲⊳ Mh,H multigrid iteration matrix‖ I − (Mh,H)−1 A ‖= C C small indepent of h⇒ mesh-width independent (h-independent) convergence



⊲⊳ multigrid methods require ahierarchy of grids

⊲⊳ this hierarchy can be cumber-some to construct

⊲⊳ example left: ΩH =???









⊲⊳ Conclusions


Algebraic Multigrid Methods

⊲⊳ Automatic construction of ΩH and AH xH = bH

⊲⊳ Using information contained in Ah only and no information on∗ differential equation∗ geometry of Ωh

⊲⊳ notion of strength of coupling between grid points exploited

⊲⊳ Strength of coupling influences the smoother

⊲⊳ Illustration for ∗ anisotropic diffusion equation∗ diffusion equation with discontinuous diffusion coefficient

M Algebraic Multigrid Methods Domenico Lahaye

Algebraic Multigrid Methods (2)

Anisotropic Diffusion Equation

⊲⊳ Equation −ǫ∂2u

∂x2− ∂2u

∂y2= f with ǫ ≪ 1 on [0,1] × [0,1]

⊲⊳ Finite difference discretization on Ωh Ah xh = bh with

[A] = 1h2

0 −1 0−ǫ 4 −ǫ0 −1 0

⊲⊳ weak coupling in x-direction, strong coupling in y-direction

⊲⊳ after point Gauss-Seidel smoothing, error is ∗ smooth in y-direction∗ oscilatory in x-direction

⊲⊳ coarsening feasable in y-direction only !

M Algebraic Multigrid Methods (2) Domenico Lahaye



00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.3

−0.2

−0.1

0

0.1

0.2

0.3

Ωh = ⇒ ΩH =



Problem with discontinuous diffusion coefficient

⊲⊳ − ∂

∂x

(ν

∂u

∂x

)− ∂

∂y

(ν

∂u

∂y

)= f with ν =

0.0

0.375

0.625

1.0

ν = 1

ν = 106

⊲⊳ after point Gauss-Seidel smoothing, error oscilatory accros interface of discon-tinuity

⊲⊳ coarsening not feasable “close to” the interface




00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

10

0.05

0.1

0.15

0.2

0.25

0.3

0.35

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

10

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Ωh = ⇒ ΩH =



⊲⊳ geometric multigrid ∗ construct hierarchy of coarser grids∗ smoother adapted to constructed hierarchy

⇒ work invested in the smoother⇒ e.g. using line smoothers

⊲⊳ algebraic multigrid ∗ simple, fixed smoother: point Gauss-Seidel∗ coarse grid correction adapted to the local

properties of the smoother⇒ work invested in the coarse grid correction



⊲⊳ Setup phase ∗ construction of splitting Ωh = Ch ∪ Fh and set ΩH = Ch

∗ construction of interpolation IhH

∗ strength of coupling between nodes coded in Ah exploited

∗ matrix dependent interpolation: (IHh )ij ∼ Ah

ij/Ahii

∗ Galerkin coarsening: AH = IHh Ah Ih

H

∗ apply recursively using AH as input

⊲⊳ Solve phase ∗ multigrid cycling



Permanent Magnet Machine Test Case



⊲⊳ Adaptive construction of triangulation in 13 refinement steps

0 2 4 6 8 10 12

104

105

Adaptive refinement step

Nod

es in

tria

ngul

atio

n



⊲⊳ Initial and intermediate triangulation



⊲⊳ Number of iterations for AMG

0 2 4 6 8 10 120

20

40

60

80

100

120

140

160


Num

ber

of it

erat

ions

V(1,1)V(2,2)W(1,1)

∗ multigrid behavior



⊲⊳ CPU-time measurements for AMG

50000 100000 150000 2000000

120

240

360

480

600

720

Number of grid points

CP

U ti

me

(in s

ecs.

)

One−level solverAMG

∗ RAMG speedup byfactor 7.5









⊲⊳ Conclusions


Krylov Subspace Methods

⊲⊳ Non-stationary iterative methods for solving A x = b

⊲⊳ Given x0 and r0 = b − A x0

find xm ∈ x0 + Km(A, r0) = spanr0, A r0, . . . , Am−1 r0

⊲⊳ Given Vm = [v1, . . . , vm] a basis for Km(A, r0) ⇒ xm = x0 + Vm ym

ym a vector of m coefficients

⊲⊳ basis Vm ∗ Arnoldi method vector ym ∗ residual projection methods∗ bi-Lanczos method ∗ residual norm minimization∗ Lanczos method

⊲⊳ No Krylov method optimal for large class of problems

M Krylov Subspace Methods Domenico Lahaye

Krylov Subspace Methods (2)

The Arnoldi Method for constructing Vm

⊲⊳ Initial vector v1 = r0/ ‖ r0 ‖

⊲⊳ Given Vm = [v1, . . . , vm] a basis for Km(A, r0)

compute A vm and orthogonalize with respect to Vm

normalize and set resulting vector equal to new basis vector vm+1

⊲⊳ recurrence relation A Vm = Vm+1 Hm+1,m Hm+1,m =

(Hm,m

0 . . .0hm+1,m

)

Hm,m upper Hessenberg

⊲⊳ long term recurrences: expensive in memory and CPU-time

M Krylov Subspace Methods (2) Domenico Lahaye


The Bi-Orthogonal Lanczos Method for constructing Vm

⊲⊳ Avoid long term recurrences using bi-orthogonality

V Tm Wm = Dm = diag[d1, . . . , dm] Wm basis for Km(AT , r∗0)

⊲⊳ recurrence relation A Vm = Vm+1 Tm+1

Tm+1 tridiagonal ⇒ short term recurrences

⊲⊳ (near) breakdown if vTi wi = 0 ( vT

i wi ∼ 0 )



Determing the vector ym

⊲⊳ xm ∈ x0 + Km(A, r0) ⇒ rm = b − A xm = r0 − A Vm ym

⊲⊳ residual projection methods ⇔ V Tm rm = 0

⇔ Hm,m ym = β1e1m × m linear system: solve by LU-factorization

⊲⊳ residual minimization methods ⇔ minimize ‖ rm ‖⇔ Hm+1,m ym = β1e1

(m + 1) × m linear system: solve inleast-squares sense



A x = b with A symmetric, positive definite

⊲⊳ A = AT ⇒ Hm,m = (Hm,m)T ⇒ Hm,m tridiagonal

⊲⊳ A pos. def. ⇒ Hm,m = V Tm A Vm pos. def. ⇒ LU-fact. without pivoting

⊲⊳ update LU-fact. from step m to m + 1

⊲⊳ coupled 2-term recurrences rm−1, pm−1, rm, pm ⇒rm+1, pm+1

⊲⊳ pTi A pj = 0 for i 6= j ⇒ conjugate gradient method (CG)



Convergence of Krylov Subspace Methods

⊲⊳ Krylov subspace methods converge superlinearly

⊲⊳ Bound for CG ‖ x − xm ‖A ≤ 2

[√cond2(A)−1√cond2(A)+1

]m‖ x − x0 ‖A

⊲⊳ Preconditioning: A x = b ⇒ M−1 A x = M−1 b

∗ M−1 A ≈ I ∗ M z = r “easy” to solve

⊲⊳ Preconditioned CG: ∀m replace rm by zm = M−1 rm

⊲⊳ Possible preconditioners for CG: MJAC , MSSOR, . . .



The Convergence of CG: A closer look

⊲⊳ CG: ∀m Tm ym = β1e1 Tm = V Tm A Vm

⊲⊳ eigenvalues of Tm are called Ritz-values

⊲⊳ Ritz-values converge to spec(Am) for increasing m

fast converges towards “outliers” in spec(Am), if anycfr. Krylov subspace methods for eigenvalue computations

⊲⊳ convergence of Ritz values governs convergence of CG








⊲⊳ Algebraic Multigrid Methods as Preconditioner

⊲⊳ Conclusions


AMG as Preconditioner

Problem with anti-periodic boundary conditions

Az|Γ1= −Az|Γ2

Number of iterationsAdaptive classical classical

refinement step AMG AMG/CG0 148 151 div 162 div 173 div 174 div 185 div 176 div 17

M AMG as Preconditioner Domenico Lahaye

AMG as Preconditioner (2)

convergence of the Ritz values towardsspec(I − M−1A ) during CG-iteration

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

AMG/CG Iteration step

Ritz

val

ues

of (

I − in

v(M

)*A

)

⊲⊳ spec(I − M−1A ) hasa few outliers

⊲⊳ outliers are captured fastby CG-iteration

M AMG as Preconditioner (2) Domenico Lahaye


Permanent Magnet Machine Test Case



⊲⊳ Number of iterations for AMG

0 2 4 6 8 10 120

20

40

60

80

100

120

140

160


Num

ber

of it

erat

ions

V(1,1)V(2,2)W(1,1)V(1,1)/CG

∗ CG accelerates andstabilizes convergence



⊲⊳ CPU-time measurements for AMG

50000 100000 150000 2000000

120

240

360

480

600

720

Number of grid points

CP

U ti

me

(in s

ecs.

)

SSOR/CGRAMGRAMG/CG

∗ AMG speedup byfactor 7.5

∗ AMG/CG speedup byfactor 15


Conclusions

⊲⊳ We presented multigrid and algebraic multigrid for two-dimensional diffusionproblems.

⊲⊳ Algebraic multigrid methods allow to solve problems on complicated geomet-ries.

⊲⊳ Algebraic multigrid methods are more efficient than single-level iterative solvers.

⊲⊳ Using algebraic multigrid as a precondioner improves its stability and speed.

M Conclusions Domenico Lahaye

Date post:	29-Sep-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Algebraic Multigrid as Solvers and as Preconditionerta.twi.tudelft.nl/.../week1_part2_slides.pdf ·...

Documents