July 1, 2010Parallel solution of the Helmholtz equation1 Parallel solution of the Helmholtz equation...

July 1, 2010 Parallel solution of the Helmholtz equation 1

Parallel solution of the Parallel solution of the Helmholtz equation with Helmholtz equation with

large wave numberslarge wave numbers

Dan GordonComputer ScienceUniversity of Haifa

Rachel GordonAerospace Eng.

Technion


Outline of TalkOutline of Talk

The Kaczmarz algorithm (KACZ) KACZ CARP (Component-

Averaged Row Projections) Applications of CARPCARP-CG: CG acceleration of CARP Sample results with the Helmholtz

equation


KACZKACZ: The Kaczmarz algorithm: The Kaczmarz algorithm

Iterative method, due to Kaczmarz (1937). Rediscovered for CT as ART

Geometric algorithm: consider the hyperplane defined by each equation

Start from an arbitrary initial point Successively project current point

onto the next hyperplane, in cyclic order


KACZ: Geometric DescriptionKACZ: Geometric Description

eq. 1

eq. 2eq. 3

initialpoint


KACZKACZ with Relaxation Parameter with Relaxation ParameterKACZ can be used with a relaxation

parameter =1: project exactly on the

hyperplane<1: project in front of hyperplane

>1: project beyond the hyperplane

Cyclic relaxation: eq. i is assigned a

relaxation parameter i


Convergence Properties of Convergence Properties of KACZKACZ

KACZ with relaxation (0<<2) converges for consistent systems:– Herman, Lent & Lutz, 1978

– Trummer, 1981

For inconsistent systems, KACZ converges cyclically:– Tanabe, 1971

– Eggermont, Herman & Lent, 1981 (for cyclic relaxation parameters).


Algebraic formulation of Algebraic formulation of KACZKACZ

Given the system Ax = b

Consider the "normal equations" system AATy = b, x=ATy

Well-known fact: KACZ is SOR applied to the normal equations

The relaxation parameter of KACZ is the usual relax. par. of SOR


Block Mode & ParallelizationBlock Mode & Parallelization

Block KACZ: projection onto affine subspace defined by a block of eqns

Block-sequential KACZ: – partition eqns into blocks

– each block consists of independent eqns

– iterate over blocks

– in each block, perform projections in parallel


CARPCARP: Component-Averaged Row Projections: Component-Averaged Row Projections

A block-parallel version of KACZ The equations are divided into blocks

(not necessarily disjoint) Initial estimate: vector x=(x1,…,xn) Suppose x1 is a variable (component

of x) that appears in 3 blocks x1 is “cloned” as y1 , z1 , t1 in the

different blocks. Perform one (or more) KACZ

iteration(s) on each block (independently, in parallel)


CARPCARP – Explanation (cont) – Explanation (cont)

The internal iterations in each block produce 3 new values for the clones of x1 : y1’ , z1’ , t1’

The next iterative value of x1 is

x1’ = (y1’ + z1’ + t1’)/3 The next iterate is

x’ = (x1’ , ... , xn’) Repeat iterations as needed for

convergence


CARPCARP as Domain Decomposition as Domain Decomposition

x xy

0 11

domain Adomain A domain Bdomain B

external gridexternal gridpoint of Apoint of A

clone of clone of x1

Note: domains may overlap


Overview of Overview of CARPCARP

domain Adomain A domain Bdomain B

KACZKACZiterationsiterations

KACZKACZiterationsiterations

averagingaveraging

cloningcloning

KACZKACZ in superspace in superspace(with cyclic relaxation)(with cyclic relaxation)


Convergence of Convergence of CARPCARP

Averaging Lemma: the component-

averaging and cloning operations of

CARP are equivalent to KACZ row-

projections in a certain superspace

(with =1) CARP is equivalent to KACZ in the

superspace, with cyclic relaxation parameters – known to converge


CARPCARP Application: Solution of stiff Application: Solution of stiff linear systems from PDEslinear systems from PDEs

Elliptic PDEs w/large convection term result in stiff linear systems (large off-diagonal elements)

CARP is very robust on these systems, as compared to leading solver/preconditioner combinations

Downside: Not always efficient


CARPCARP Application: Electron Tomography Application: Electron Tomography(joint work with J.-J. Fernández)(joint work with J.-J. Fernández)

3D reconstructions: Each processor is assigned a block of consecutive slices. Data is in overlapping blobs.

The blocks are processed in parallel. The values of shared variables are

transmitted between the processors which share them, averaged, and redestributed.


CARP-CGCARP-CG: CG acceleration of : CG acceleration of CARPCARP

CARP is KACZ in some superspace (with cyclic relaxation parameters)

Björck & Elfving (BIT 79): developed CGMN, which is a (sequential) CG-acceleration of KACZ (double sweep, fixed relax. parameter)

We extended this result to allow cyclic relaxation parameters

Result: CARP-CG


CARP-CGCARP-CG: Properties: Properties

Same robustness as CARP Very significant improvement in

performance on stiff linear systems derived from elliptic PDEs

Very competitive runtime compared to leading solver/preconditioner combinations on systems derived from convection-dominated PDEs

Improved performance in ET


CARP-CGCARP-CG: Properties: Properties

On one processor, CARP-CG is identical to CGMN

Particularly useful on systems with LARGE off-diagonal elements– example: convection-dominated PDEs

Discontinuous coefficients are handled without requiring domain decomposition (DD)


Robustness of CARP-CG

KACZ inherently normalizes the eqns After normalization, the diagonal

elements of AAT are larger than the off-diagonal ones (in each row)

This is not diagonal dominance, but it makes the normal eqns manageable

Normalization was also found to be useful for discontinuous coefficients


The Helmholtz Equation

Eqn: -Δu - k2u = f Wave length: = 2/k No. of grid pts per : Ng = 2/kh Shifted Laplacian approach:

– Bayliss, Goldstein & Turkel, 1983– Erlangga, Vuik & Oosterlee, 2004/06

-Δu – (ik2 u = fuses multigrid to solve the PC (PC = preconditioner)


The Helmholtz Equation Bollhöfer, Grote & Schenk, 2009:

introduced algebraic multilevel PC for the Helmholtz eqn in heterogeneous media. Uses symmetric max weight matchings and an inverse-based pivoting method.

Apologies to many other contributors to this problem!


Experiments

CARP-CG was used with a fixed relaxation parameter of 1.7 in all cases

Domain: unit square []

2nd order central difference scheme


Problem 1 (with analytic sol'n)

Based on Erlangga et al '04, §6.1 Eqn: (Δ+k2)u = (k2–52)sin(x)sin(2y) bndry condition: u=0 on all sides Analytic solution: u = sin(x)sin(2y) Grid points per : Ng = 6,8,10,12 No. of processors: 1 – 32 k = 100, 300


Problem 2 (homogeneous) Based on Erlangga et al '04, §6.2 Eqn: Δu + k2u = 0 Domain: unit square [0,1]x[0,1] Dirichlet bndry cond. on one side, with a

discontinuity at midpoint 1st-order absorbing bndry cond. on other

sides Grid points per : Ng = 6, 8, 10 No. of processors: 1 – 32 k = 75, 150, 300, 450, 600




Problem 3 (heterogeneous) 3-layer heterogeneous problem Based on Erlangga et al '04, §6.3 Everything is identical to Problem 2 EXCEPT:

k=600

k=450

k=300





Comparative Timing Results

Methodtime/iter

1 proc

time/iter

16 proc

it-ratio

1 proc

it-ratio

16 proc

CARP-CG 0.0978 0.0134 1.00 1.00

Bi-CGSTAB 0.1477 0.0344 1.51 2.56

GMRES(restart=

10)0.1490 0.0212 1.52 1.58

Time/iter of Bi-CGSTAB and GMRES relative to CARP-CG

it-ratio = (time/iter of algorithm) / (time/iter of CARP-CG)

Results from CARP-CG paper (PARCO, 2010)

Timing and Speedup Results

No. proc No. Iter Time (s) Speed-up Efficiency

1 7025 1256 1.00 100%

2 7039 822 1.53 76.4%

4 7066 462 2.72 68.0%

8 7115 255 4.92 61.5%

15 7206 159 7.92 52.8%

Problem 2, k=600, Ng=8, grid: 763763

582,169 (complex) equations, rel-res<10-7

Summary

CARP-CG is highly scalable on the Helmholtz eqn w/high wave numbers

Applicable to discontinuous coefficients

Very simple to implement General-purpose – useful also for

other problems with large off-diagonal elements and discontinuous coefficients


Other Potential ApplicationsOther Potential Applications Fourth-order schemes for the

Helmholtz equation (already have good initial results)

Maxwell equations Saddle-point problems Circuit problems Linear solvers in some eigenvalue

methods ...


Relevant PublicationsRelevant Publicationshttp://cs.haifa.ac.il/~gordon/pub.html

CARP: SIAM J Sci Comp 2005

CGMN: ACM Trans Math Software 2008

Microscopy: J Parallel & Distr Comp 2008

Large convection + discontin coef: CMES 2009

CARP-CG: Parallel Comp 2010

Scaling for discont coef: J Comp & Appl Math 2010

CARP-CG SOFTWARE AVAILABLE ON REQUEST

THANK YOU!

Date post:	25-Dec-2015
Category:	Documents
Upload:	lucy-gray
View:	219 times
Download:	3 times

July 1, 2010Parallel solution of the Helmholtz equation1 Parallel solution of the Helmholtz equation...

Documents