Parallel solution of the Helmholtz equation with large wave numbers

July 1, 2010 Parallel solution of the Helmholtz equation 1

Parallel solution of the Parallel solution of the Helmholtz equation with Helmholtz equation with

large wave numberslarge wave numbers

Dan GordonComputer ScienceUniversity of Haifa

Rachel GordonAerospace Eng.

Technion


Outline of TalkOutline of Talk The Kaczmarz algorithm (KACZ) KACZ CARP (Component-

Averaged Row Projections) Applications of CARPCARP-CG: CG acceleration of CARP Sample results with the Helmholtz

equation


KACZKACZ: The Kaczmarz algorithm: The Kaczmarz algorithm

Iterative method, due to Kaczmarz (1937). Rediscovered for CT as ART

Geometric algorithm: consider the hyperplane defined by each equation

Start from an arbitrary initial point Successively project current point

onto the next hyperplane, in cyclic order


KACZ: Geometric DescriptionKACZ: Geometric Description

eq. 1

eq. 2eq. 3

initialpoint


KACZKACZ with Relaxation Parameter with Relaxation ParameterKACZ can be used with a relaxation

parameter =1: project exactly on the

hyperplane<1: project in front of hyperplane>1: project beyond the hyperplane Cyclic relaxation: eq. i is assigned a

relaxation parameter i


Convergence Properties of Convergence Properties of KACZKACZ

KACZ with relaxation (0<<2) converges for consistent systems:– Herman, Lent & Lutz, 1978– Trummer, 1981

For inconsistent systems, KACZ converges cyclically:– Tanabe, 1971– Eggermont, Herman & Lent, 1981 (for

cyclic relaxation parameters).


Algebraic formulation of Algebraic formulation of KACZKACZ Given the system Ax = b Consider the "normal equations"

system AATy = b, x=ATy Well-known fact: KACZ is SOR

applied to the normal equations The relaxation parameter of KACZ is

the usual relax. par. of SOR


Block Mode & ParallelizationBlock Mode & Parallelization Block KACZ: projection onto affine

subspace defined by a block of eqns Block-sequential KACZ:

– partition eqns into blocks– each block consists of independent eqns– iterate over blocks– in each block, perform projections in

parallel


CARPCARP: Component-Averaged Row Projections: Component-Averaged Row Projections

A block-parallel version of KACZ The equations are divided into blocks

(not necessarily disjoint) Initial estimate: vector x=(x1,…,xn) Suppose x1 is a variable (component

of x) that appears in 3 blocks x1 is “cloned” as y1 , z1 , t1 in the

different blocks. Perform one (or more) KACZ

iteration(s) on each block (independently, in parallel)


CARPCARP – Explanation (cont) – Explanation (cont) The internal iterations in each block

produce 3 new values for the clones of x1 : y1’ , z1’ , t1’

The next iterative value of x1 is x1’ = (y1’ + z1’ + t1’)/3

The next iterate isx’ = (x1’ , ... , xn’)

Repeat iterations as needed for convergence


CARPCARP as Domain Decomposition as Domain Decomposition

x x

y0 1

1

domain Adomain A domain Bdomain B

external gridexternal gridpoint of Apoint of A

clone of clone of x1

Note: domains may overlap


Overview of Overview of CARPCARP

domain Adomain A domain Bdomain B

KACZKACZiterationsiterations

KACZKACZiterationsiterations

averagingaveraging

cloningcloning

KACZKACZ in superspace in superspace(with cyclic relaxation)(with cyclic relaxation)


Convergence of Convergence of CARPCARP Averaging Lemma: the component-

averaging and cloning operations of CARP are equivalent to KACZ row-projections in a certain superspace (with =1)

CARP is equivalent to KACZ in the superspace, with cyclic relaxation parameters – known to converge


CARPCARP Application: Solution of stiff Application: Solution of stiff linear systems from PDEslinear systems from PDEs

Elliptic PDEs w/large convection term result in stiff linear systems (large off-diagonal elements)

CARP is very robust on these systems, as compared to leading solver/preconditioner combinations

Downside: Not always efficient


CARPCARP Application: Electron Tomography Application: Electron Tomography(joint work with J.-J. Fernández)(joint work with J.-J. Fernández)

3D reconstructions: Each processor is assigned a block of consecutive slices. Data is in overlapping blobs.

The blocks are processed in parallel. The values of shared variables are

transmitted between the processors which share them, averaged, and redestributed.


CARP-CGCARP-CG: CG acceleration of : CG acceleration of CARPCARPCARP is KACZ in some superspace

(with cyclic relaxation parameters) Björck & Elfving (BIT 79): developed

CGMN, which is a (sequential) CG-acceleration of KACZ (double sweep, fixed relax. parameter)

We extended this result to allow cyclic relaxation parameters

Result: CARP-CG


CARP-CGCARP-CG: Properties: Properties Same robustness as CARP Very significant improvement in

performance on stiff linear systems derived from elliptic PDEs

Very competitive runtime compared to leading solver/preconditioner combinations on systems derived from convection-dominated PDEs

Improved performance in ET


CARP-CGCARP-CG: Properties: Properties On one processor, CARP-CG is

identical to CGMN Particularly useful on systems with

LARGE off-diagonal elements– example: convection-dominated PDEs

Discontinuous coefficients are handled without requiring domain decomposition (DD)


Robustness of CARP-CGKACZ inherently normalizes the eqns After normalization, the diagonal

elements of AAT are larger than the off-diagonal ones (in each row)

This is not diagonal dominance, but it makes the normal eqns manageable

Normalization was also found to be useful for discontinuous coefficients


The Helmholtz Equation Eqn: -Δu - k2u = f Wave length: = 2/k No. of grid pts per : Ng = 2/kh Shifted Laplacian approach:

– Bayliss, Goldstein & Turkel, 1983– Erlangga, Vuik & Oosterlee, 2004/06

-Δu – (ik2 u = fuses multigrid to solve the PC (PC = preconditioner)


The Helmholtz Equation Bollhöfer, Grote & Schenk, 2009:

introduced algebraic multilevel PC for the Helmholtz eqn in heterogeneous media. Uses symmetric max weight matchings and an inverse-based pivoting method.

Apologies to many other contributors to this problem!


ExperimentsCARP-CG was used with a fixed

relaxation parameter of 1.7 in all cases

Domain: unit square []

2nd order central difference scheme


Problem 1 (with analytic sol'n)

Based on Erlangga et al '04, §6.1 Eqn: (Δ+k2)u = (k2–52)sin(x)sin(2y) bndry condition: u=0 on all sides Analytic solution: u = sin(x)sin(2y) Grid points per : Ng = 6,8,10,12 No. of processors: 1 – 32 k = 100, 300


Problem 2 (homogeneous) Based on Erlangga et al '04, §6.2 Eqn: Δu + k2u = 0 Domain: unit square [0,1]x[0,1] Dirichlet bndry cond. on one side, with a

discontinuity at midpoint 1st-order absorbing bndry cond. on other

sides Grid points per : Ng = 6, 8, 10 No. of processors: 1 – 32 k = 75, 150, 300, 450, 600




Problem 3 (heterogeneous) 3-layer heterogeneous problem Based on Erlangga et al '04, §6.3 Everything is identical to Problem 2 EXCEPT:

k=600

k=450

k=300





Comparative Timing Results

Methodtime/iter1 proc

time/iter16 proc

it-ratio1 proc

it-ratio16 proc

CARP-CG 0.0978 0.0134 1.00 1.00

Bi-CGSTAB 0.1477 0.0344 1.51 2.56

GMRES(restart=

10)0.1490 0.0212 1.52 1.58

Time/iter of Bi-CGSTAB and GMRES relative to CARP-CGit-ratio = (time/iter of algorithm) / (time/iter of CARP-CG)

Results from CARP-CG paper (PARCO, 2010)

Timing and Speedup Results

No. proc No. Iter Time (s) Speed-up Efficiency1 7025 1256 1.00 100%2 7039 822 1.53 76.4%4 7066 462 2.72 68.0%8 7115 255 4.92 61.5%15 7206 159 7.92 52.8%

Problem 2, k=600, Ng=8, grid: 763763 582,169 (complex) equations, rel-res<10-7

SummaryCARP-CG is highly scalable on the

Helmholtz eqn w/high wave numbers Applicable to discontinuous

coefficients Very simple to implement General-purpose – useful also for

other problems with large off-diagonal elements and discontinuous coefficients


Other Potential ApplicationsOther Potential Applications Fourth-order schemes for the

Helmholtz equation (already have good initial results)

Maxwell equations Saddle-point problems Circuit problems Linear solvers in some eigenvalue

methods ...


Relevant PublicationsRelevant Publicationshttp://cs.haifa.ac.il/~gordon/pub.htmlCARP: SIAM J Sci Comp 2005CGMN: ACM Trans Math Software 2008Microscopy: J Parallel & Distr Comp 2008Large convection + discontin coef: CMES 2009CARP-CG: Parallel Comp 2010Scaling for discont coef: J Comp & Appl Math 2010

CARP-CG SOFTWARE AVAILABLE ON REQUEST

THANK YOU!

Date post:	22-Feb-2016
Category:	Documents
Upload:	odessa
View:	26 times
Download:	0 times

Parallel solution of the Helmholtz equation with large wave numbers

Documents