Linear solvers & preconditioners · Mesh & matrixSparse adressing...

Mesh & matrix Sparse adressing Preconditioner Krylov-solvers CG BiCG Tutorial:PBiCGSTAB

Linear solvers & preconditionersGOFUN 2018, Braunschweig

Dr. Thorsten Grahs

Feb. 22, 2018

move http://www.move-csc.de | Dr. Thorsten Grahs — Linear solvers & preconditioners | 1/77

http://www.move-csc.de


Matrix solver

In this talk we cover

� Assembling the matrix

� Correspondence betweenmesh & matrix

� Class lduMatrix

� Solver in OpenFOAM

� Krylov sub-space methods

� BiCG method

� Tutorial BiCGSTAB




Matrix solver

� Results directly from the discretisation of the governing equa-tions

� Assembled as� fvScalarMatrix� fvVectorMatrix� fvTensorMatrix

� ExamplefvVectorMatrix UEqn

(

fvm::ddt(U)

+ fvm::div(phi, U)

- fvm::laplacian(nu, U)

);




Matrix solver

� An instance of the class� fvScalarMatix (e.g. for pEqn) or� fvVectorMatrix (e.g. for UEqn)

is built, which represents the matrix(and also rhs, preconditioner and solver)

� The coefficients results from the discretisation of the fvm oper-ators

� diagonal entries f.i. from the temporal discretisation� off-diagonal entries from the fluxes from the neighbouring cells� Right hand side from source terms (fvc-operators) or old time

steps




Solving an equation

� Solving an equation in OpenFOAMalways follows the same pattern:

� Initialisation� Assembling the matrix� Solving the system� Correcting the solution

� In the following, we consider the approachexemplified by the solver icoFoam




Matrix creation & initialisationExample icoFoam

#include "fvCFD.H"

// * * * * * * * * * * * * * * * * * * * *//

int main(int argc, char *argv[])

{

#include "setRootCase.H"

#include "createTime.H"

#include "createMesh.H"

#include "createFields.H"

#include "initContinuityErrs.H"

Info<< "\nStarting time loop\n" << endl;

while (runTime.loop())

{

Info<< "Time = " << runTime.timeName()

<< nl << endl;

#include "readPISOControls.H"

#include "CourantNo.H"

fvVectorMatrix UEqn

(

fvm::ddt(U)

+ fvm::div(phi, U)

- fvm::laplacian(nu, U)

);

solve(UEqn == -fvc::grad(p));

� Creating the instancefvVectorMatrix UEqn fromthe class fvVectorMatrix

� Initialisation withfvm::ddt(U) + fvm::div(phi,U) -

fvm::laplacian(nu,U)

� All operators are from fvm

namespace (implicit), i.e. all con-tribute to the matrix

� Solved with rhs -fvc::grad(p) (ex-plicit)

� Correcting the solution with thePISO-Loop




Mesh & matrix structure

Correspondence

Mesh ⇔ Au = b

� Obviously, there is a strong correspondence betweenmesh and matrix

� The number of cells corresponds with therank n of the matrix A

� And so naturally with the dimension of the solution vector u.

� The entries of the matrix stems from theweights of the numerical schemes

� In general, A is a sparse matrix




Mesh example




Relation Matrix & Mesh

� The number of diagonal elements equals the number of cells ofthe mesh.

� The number of non-zero upper and lower elements equal thenumber of internal faces of the mesh.

� The upper matrix contains values belonging to the owners offaces.

� The lower matrix contains values belonging to the neighbourcells.

� Off-diagonal matrix element contain a non-zero value if a faceis shared between an owner and neighbour cell on that position(in the mesh)




Matrix structureMatrix according to the mesh example

� 10 volumes/cells

� 36 nodes/corners

� 47 faces, 13 internal faces

a0,0 a0,1 0 0 0 a0,5 0 0 0 0a1,0 a1,1 a1,2 0 0 0 a1,6 0 0 0

0 a2,1 a2,2 a2,3 0 0 0 a2,7 0 00 0 a3,2 a3,3 a3,4 0 0 0 a3,8 00 0 0 a4,3 a4,4 0 0 0 0 a4,9

a5,0 0 0 0 0 a5,5 a5,6 0 0 00 a6,1 0 0 0 a6,5 a6,6 a6,7 0 00 0 a7,2 0 0 0 a7,6 a7,7 a7,8 00 0 0 a8,3 0 0 0 a8,7 a8,8 a8,9

0 0 0 0 a9,4 0 0 0 a9,8 a9,9




lduMatrixMatrix according to the mesh example

� The system, resulting from the FV discretisation, is stored astype lduMatrix

� fvMatrix is a sub class from lduMatrix

� In general we deal with sparse matrices.� The non-zero coefficients are stored in three arrays:

� lower()� diag()� upper()

� For our example we have the following entries for the arrays:

lower() = (a1,0, a5,0, a2,1, a6,1, a3,2, a7,2, a4,3, a8,3, a9,4, a6,5, a7,6, a8,7, a9,8)

diag() = (a0,0, a1,1, a2,2, a3,3, a4,4, a5,5, a6,6, a7,7, a8,8, a9,8)

upper() = (a0,1, a0,5, a1,2, a1,6, a2,3, a2,7, a3,4, a3,8, a4,9, a5,6, a6,7, a7,8, a8,9)




Run orderMatrix according to the mesh example

� The order of accessing matrix elements in OpenFOAM is asfollows:

lower(), diag(), upper()




Addressing

� The addressing for the non-zero coefficients are stored in threelists of the class lduAdressing:

� lowerAddr()

Keeps the row number of the according element in array lower()and the column number for array upper()

� upperAddr()

Vice versa, i.e. keeps column number for lower() and rownumber for upper() entries

� ownerStartAddr()Keeps the information, at which position in

� upper() a new row starts� lower() a new column starts




AddressingMatrix according to the mesh example

� The address arrays for the example matrix looks as follows:

� upperAddr()

= (0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 8)� lowerAddr()

= (1, 5, 2, 6, 3, 7, 4, 8, 9, 6, 7, 8, 9)� ownerStartAddr() = (0, 2, 4, 6, 8, 9, 10, 11, 12)

� lduMatrix

� lower(), diag(), upper() and lduAdressing

are function of the matrix class lduMatrix.� lduMatrix has further functions for solving algorithms and statis-

tics i.e. solver, solverPerformance, smoother & preconditioners

� Class definition can be found at$FOAM SRC/OpenFOAM/matrices/lduMatrix




Linear system solver functionsSolver sub-classes in lduMatrix

� lduMatrix::solverPerformance (solver statistics)Class which is returned by each matrix solver. Keeps informa-tions about the solver characteristics, e.g.

� solver name� field/variable name� Initial residual� Final residual� Number of iterations

� Sub-classes solver, smoother & preconditioner are virtualbase classes:

� They define an interface, which has to be implemented by aconcrete matrix solver

� This allows run-time selection for the matrix solver� An implementation of a concrete matrix solver algorithm has to

be derived from the class solver




Matrix manipulation

� fvMatrix defines function for accessing matrix elements� tmp<volScalarField> D() returns diagonal elements� tmp<volScalarField> A() returns diagonal elements weighted

with inverse of cell volume� tmp<volScalarField> H() returns off-diagonal elements

� The assembled matrix correspondence directly with the fvm

operators

� This enables the construction of the matrix system directly fromthe implementation of the equation in top-level code

fvm :: ddt(U) + fvm :: div(phi, U)⇒ Au = b




Test case

� Consider heat conduction in a thin rod

� The temperature distribution is governed by a Laplace-equation(From Fourier’s Law)

d

dx

(kdT

dx

)= 0

� Diameter D = 0.001m2, Conductivity k = 1000Wm−1K−1




Test caseExpectations

What do we expect?

� Linear distribution inside the rod.




myLaplacianFoam

� We change the original OpenFOAM solver laplacianFoamto a customized (simplified) version: myLaplacianFoam

� Clone solver> run

> mkdir programming

> cd programming

> cp -r $FOAM_SOLVERS/basic/laplacianFoam myLaplacianFoam

� Adapt source code & make-filesin myLaplacianFoam/Make/files:laplacianFoam.C

EXE = $(FOAM_APPBIN)/laplacianFoam⇒ mylaplacianFoam.C

EXE = $(FOAM_USER_APPBIN)/mylaplacianFoam




Adapt codemyLaplacianFoam.c

� Change name

> mv laplacianFoam.C myLaplacianFoam.C

� In myLaplacianFoam.C changewhile (simple.loop())

{

Info<< "Time = " << runTime.timeName()

<< nl << endl;

while (simple.correctNonOrthogonal())

{

solve

(

fvm::ddt(T)

- fvm::laplacian(DT, T)

);

}

#include "write.H"

Info<< "ExecutionTime = "

<< runTime.elapsedCpuTime()

<< " s"

<< " ClockTime = "

<< runTime.elapsedClockTime()

<< " s"

<< nl << endl;

}

⇒


{

fvScalarMatrix

TEqn(fvm::laplacian(k, T));

TEqn.solve();

forAll(T, cellI)

{

Info<< "X = " <<

mesh.C()[cellI].component(vector::X)

<< ", T = " << T[cellI] << endl;

}

runTime.writeAndEnd();

}




Adapt codecreateFields.C

� In create.C changeInfo<< "Reading diffusivity DT\n" << endl;

dimensionedScalar DT

(

transportProperties.lookup("DT")

);

⇒Info<< "Reading diffusivity k\n" << endl;

dimensionedScalar k

(

transportProperties.lookup("k")

);

� Compile the modified code:> cd myLaplacianFoam

> wmake

� Run the test case> cd rodTemperature

> blockMesh

> myLaplacianFoam

...

Calculating temperature distribution

DICPCG: Solving for T,Initial residual = 1,Final residual = 1.21266e-16,No Iterations 1

X = 0.05, T = 140

X = 0.15, T = 220

X = 0.25, T = 300

X = 0.35, T = 380

X = 0.45, T = 460

End




Result

Result in accordance with our expectations




Extension

Change in myLaplacianFoam.C#include "fvCFD.H"

#include "simpleMatrix.H"

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

int main(int argc, char *argv[])

{

#include "setRootCase.H"

#include "createTime.H"

#include "createMesh.H"

#include "createFields.H"

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

Info<< "\nCalculating temperature distribution\n" << endl;


{

fvScalarMatrix TEqn(fvm::laplacian(k, T));

TEqn.solve();

forAll(T, cellI)

{

Info<< "X = " <<

mesh.C()[cellI].component(vector::X)

<< ", T = " << T[cellI] << endl;

}

#include "writeMatCoeffs.H"




Adapt codeCreate writeMatCoeffs.C

� Investigating the coefficient matrix

label NC = mesh.nCells(); //Number of cells

simpleMatrix<scalar> A(NC); //Coeff.matrix

// Initialization of matrix

for(label i=0; i<NC; i++)

{

A.source()[i] = 0.0;

for(label j=0; j<NC; j++)

{

A[i][j] = 0.0;

}

}

// Assigning diagonal coefficients


{

A[i][i] = TEqn.diag()[i];

}

// Assigning off-diagonal coefficients

for(label faceI=0;

faceI<TEqn.lduAddr().lowerAddr().size(); faceI++)

{

label l = TEqn.lduAddr().lowerAddr()[faceI];

label u = TEqn.lduAddr().upperAddr()[faceI];

A[l][u] = TEqn.upper()[faceI];

A[u][l] = TEqn.upper()[faceI];

}

// Assigning contribution from BC

forAll(T.boundaryField(), patchI)

{

const fvPatch &pp =

T.boundaryField()[patchI].patch();

forAll(pp, faceI)

{

label cellI = pp.faceCells()[faceI];

A[cellI][cellI]

+= TEqn.internalCoeffs()[patchI][faceI];

A.source()[cellI]

+= TEqn.boundaryCoeffs()[patchI][faceI];

}

}

Info << "\n==Coefficients of Matrix A=="

<< endl;


{

for(label j=0; j<NC; j++)

{

Info<< A[i][j] << " ";

}

Info<< A.source()[i] << endl;

}

Info<< "\n==> Solution: " << A.solve()

<< endl;




Running adapted code

� Recompile and rerun test case rodTemperature

Solver outputCalculating temperature distribution

DICPCG: Solving for T, Initial residual = 1, Final residual = 1.21266e-16, No Iterations 1

X = 0.05, T = 140

X = 0.15, T = 220

X = 0.25, T = 300

X = 0.35, T = 380

X = 0.45, T = 460

==Coefficient Matrix==

-300 100 0 0 0 -20000

100 -200 100 0 0 0

0 100 -200 100 0 0

0 0 100 -200 100 0

0 0 0 100 -300 -100000

Solution: 5(140 220 300 380 460)

End




Linear systems

Iterative solvers� The system of equation in OpenFOAM is solved by iterative

matrix algorithms� Based on the starting vector x0 the iteration rule

xk+1 = Axk

is applied� The iteration stops, when the residual is smaller than a prescribed

tolerance tol:

‖Axk+1 − b‖ < tol

� Sometimes one tries to accelerate the solution behaviour of theiterative process.

� This goal can be achieved by pre-conditioning




Preconditioning

Goal

� Enhance matrix properties in order to optimize condition number(cond(A) close to 1)

⇒ faster convergence of the solver

� One is looking for a matrix M with

M−1Ax = M−1b

and M easy to invert.

� The easiest choice would be M = I(To simple - yields just the original system)

� The best choice would be M = A−1

(To costly to invert A)




Preconditioner in OpenFOAM

� diagonalPreconditioner

for symmetric & nonsymmetric matrices (not very effective)

� DICPreconditioner

Diagonal Incomplete Cholesky preconditionerfor symmetric matrices

� DILUPreconditioner

Diagonal Incomplete LU preconditionerfor nonsymmetric matrices

� FDICPreconditioner

Fast Diagonal Incomplete Cholesky preconditioner

� GAMGPreconditioner

Geometric Agglomerated algebraic MultiGrid preconditioner

� noPreconditioner




Preconditioners

� Diagonal preconditioner M = DUses the diagonal as preconditioner. Not very efficient.

� Incomplete LU preconditioner A = LU + R, M = LU� Apply Gaussian elimination algorithm, but only on allowed pat-

tern ⇒ incomplete LU factorization called ILU.� Reduce in the for-loops the indices to the indices with

– allowed pattern, e.g. ILU(0) for pattern of A– values that are not too small, ILUT for ILU with treshold

� Leads to approximate LU factorization

A = LU + R, preconditioner M = LU

with all ignored fill-in entries collected in R.

� Incomplete Cholesky preconditioner A = GGT +R, M = GGT

Similar to ILU, but based on incompleted Cholesky factorization




Matrix solver in OpenFOAM

� BICCG

Diagonal incomplete LU preconditioned BiCG solver� diagonalSolver

Solver for symmetric and nonsymmetric matrices� GAMG

Geometric Agglomerated algebraic Multi-Grid solver� ICC

Incomplete Cholesky Conjugate Gradient solver� PBiCG

Bi-Conjugate Gradient solver with preconditioner� PCG

Conjugate Gradient solver with preconditioner� smoothSolver

Iterative solver with run-time selectable smoother

Most of these solvers are Krylov-subspace solvers




Krylov subspace methods

� A Krylov1 subspace method is an iterative matrix algorithm forsolving the linear system Ax = b of the form:

xm ∈ x0+Km, Km = KM(A, r0) = span(r0,Ar0, . . . ,Am−1r0)

with

r0 = b− Ax0, (b− Axm) ⊥ Lm, Km, Lm ∈ Rm ⊂ Rn

� Km represents the Krylov subspace where the solution is sought.

Why is this desirable?

1Aleksey Nikolaevich Krylov (1863 - 1945), russian naval engineer and mathe-matician




Krylov methodMotivation

� Idea of a Krylov method stems from the fact that� the inverse of a N × N non-singular Matrix A can be expressed

by a polynomial in A of degree (N − 1).� i.e.

A−1 = c0I + c1A + c2A2 + . . .+ cN−1A

N−1

� This is a direct conclusion from the Cayley-Hamilton theorem

� Since the true solution x? is

x? = A−1b = c0b+ c1Ab+ c2A2b+ . . .+ cN−1A

N−1b = p(A)

with polynomial p(A) =∑N−1

i=0 ciAib

� Hence we can approximate the solution x? = A−1b by applyinga polynomial with m� n

� We only need matrix-vector multiplication!




Krylov methodThe iteration approach

Krylov subspace methods for the linear system

Ax = b

attempt to approximate the solution x = A−1b with

xj ∈ x0 + Kj , Kj(A, r0), r0 = b− Ax0.

The specific choice of the polynomial (i.e. the coefficients) dependson the concrete method.




Krylov methodsRemarks

� If A has a dominant eigenvector, then it is likely that

Wj = [b,Ab,A2b, . . . ,Aj−1b]

is very, very ill conditioned for large values of j .

� In general, one needs orthonormal vectors {vi}ji=0 such that

Kj(A,b) = span[v0, v1, . . . , vj ]

in order to avoid numerical problems.

� The vectors vi can be computed using an standard orthogonal-ization approach, f.i. the Arnoldi algorithm.




Krylov methodsStandard methods

� CG: The Conjugate Gradient algorithm applies to systemswhere A is symmetric positive definite (SPD)

� GMRES: The Generalized Minimal RESidual algorithm is thefirst method to try if A is not SPD.

� BiCG: The BiConjugate Gradient algorithm applies to generallinear systems, but the convergence can be quite erratic.

� BiCGstab: The stabilized version of the BiConjugate Gradientalgorithm.

Remark

� We focus her on CG and BiCG. In the tutorial we show how toextend BiCG to BiCGstab.




The Conjugated Gradient Method (CGM)

� CG is the most popular iterative method for solving large systemsof linear equation of the form

Ax = b

where A ∈ Rn×n be symmetric positive definite and b ∈ Rn.

� If A would be dense, the best way would be to factor(LU, QR,..) and solve by back-substitution

� For sparse systems CG is your method.




Conjugated Gradients

� Associated with the system Ax = b we have a quadratic function,f , given by

f (x) :=1

2xTAx− xTb

� The most instructive explanation of the CG method starts fromthe fact, that (if A is symmetric positive definite) the solutionof

Ax = b

minimizes the quadratic function f .

Why????





� To this uses we calculate the first and second order derivativesof f :

∇f (x) = Ax− b

Hf (x) = A

with� the gradient ∇f = (∂1f , . . . , ∂nf )� the Hessian Hf = (∂2

ij f )ij .

� Since all higher order derivatives vanish, we can rewrite f as itsTaylor polynomial.




Taylor polynomial

� We rewrite f as its Taylor polynomial of degree two, i.e.

f (x) = f (y)− (x− y)T∇f (y) +1

2(x− y)THf (y)(x− y)

= f (y)− (x− y)T (Ay − b) +1

2(x− y)TA(x− y)

≥ f (y)− (x− y)T (Ay − b)

� In the last step, we have used that A is positive definite.

� If y = x∗ solves Ax = b⇒ f (x) ≥ f (x∗), i.e. x∗ minimises f .

� On the other hand, every minimiser y of f has to satisfy thenecessary condition 0 = ∇f (y) = Ay − b.

� This is only possible for y = x∗




Example: Graph, Contours Gradient of f

� Here we show the graph ofa quadratic form f .

� The minimum point of thissurface is x with Ax = b.

� Contours of f are ellipsoidals(i.e. f=const.)




Constructing an iterative method

� Now we construct an iterative method which tries to find theminimum of f .

� We start from the iteration rule

xj+1 = xj + αjpj

with search direction pj and step length αj

Question?

� How to chose α.

� How to choose pj




Determine step length α

Assume for a moment that we already have chosen the search direc-tion pj .

� With given pj the best possible step-length can be determinedby finding the minimum of f along xj + αpj .

� Setting ψ(α) = f (xj + αpj) the necessary condition for a mini-mum is

0 = ψ′(α) = pTj ∇f (xj + αpj)

= pTj [A(xj + αpj)− b]

= pTj Axj + αpTj Apj − pTj b.




Step-length αj

Theorem

Assume that the search direction pj 6= 0 has already been determined.Then we can compute the optimal step-length α for the new iteration

xj+1 = xj + αjpj

by

αj =pTj (b− Axj)

pTj Apj=

pTj rj

pTj Apj(1)

� Thus, we have to determine the search direction.




Search direction – 1st try

� Use the unity vectors ei as search directions, i.e.

p0 = e1,p1 = e2, . . . ,pn−1 = en,pn = e1,pn+1 = e2, . . .

� Than we have

eTi Aei = aii and eTi (Ax− b)ei =n∑

j=1

aijxj − bi

� Inserting (1) – the definition for α – yields

xkk = xk−1 + αk−1pk−1 = xk−1 − 1

akk

n∑j=1

akjxk−1j − bl

ek

for k = 0, 1, . . . , n − 1.




Search direction – 1st try

Problematic:

� Only the k-th component of the vector xk is updated.

� If we consider one cycle, we get

xk =1

akk

bk −n∑

j<k

akjxkj −

∑akjx

0j

,

xki = xk−1i , k 6= i

� Therefore, one cycle is one Gauss-Seidel iteration.




Search direction – 2nd try

� One natural way to choose the search direction is the directionof the steepest descent.

� This is given by the negative gradient of the target function:

pj = −∇f (xj)

= −(Axj − b)

= rj




Steepest Descent

Algorithm

Choose x1 and set p1 =b− Ax1

for j = 1, 2, . . . do

αj =pTj rj

pTj Apjxj+1 = xj + αjpjpj+1 = b−Axj+1

end




Orthogonal search direction

� Notice that, by choosing αj and pj as constructed, we have

0 = pTj (Axj+1 − b) = −pTj pj+1

� Thus, the derived search directions are orthogonal

� Furthermore, there are successive orthogonal.� On a first glance ...

� orthogonal search direction are very good.

� On a second view� steepest descent converges rather slowly.




Steepest descent method

Bad if λmax/λmin is large.

� Consider Ax = b with

A =

(λ1 00 λ2

), b =

(λ1

λ2

).

� Use start vector x0 = (−9, 1)T . The solution is (1, 1)T .

� Looking for ε = ‖rk‖ < 10−4:




Example

Example Let us consider the following setting:

A =

(1 00 9

), b =

(00

), x =

(91

).

Convergence history:

xj = (0.8)j−1

(9

(−1)j−1

)




Search direction 3rd approach

Idea:

� Determine search direction pk such, that pk is A-conjugatedw.r.t. the previous directions p0,p1, . . .pk+1

Definition

Two vectors x, y ∈ Rn are calledA-conjugated

ifxTAy = 0

holds.




A-conjugates search directions

Level curves of f (x) = xTAx = 12 (x2

1 + 9x22 )

� The level curses of f are el-lipses.

� The orthogonality forces thesearch direction tangentialto the level sets

� If we introduce A-conjugatedsearch directions, i.e. sj =A1/2pj one finds the mini-mum in two steps.

We have 0 = sTj sj+1 = pTj Apj+1




Conjugated Gradient (CG) method

Theorem

Assume p0,p1, . . . ,pn−1 6= 0 are pairwise A-conjugated vectors.Then, the scheme

xk+1 = xk + αkpk

with

αk = − (rk)T rk

(pk)TApk

converges in at most n steps against the exact solution.





Algorithm: CG method (Hestenes & Stiefel 1952)

Choose x0

Set p0 = r0 = b− Ax0 and j = 0.while ‖rj‖2 > ε do

vj = Apj , αj =rTj rj

vTj pjxj+1 = xj + αjpjrj+1 = rj − αjvj

βj =‖rj+1‖2

2

‖rj‖22

pj+1 = rj+1 + βjpjj = j + 1

end





Remarks

� It can be proved, that the vectors pk are pair-wise A-conjugated.

� Theoretically, after at most n steps the solution can be computed.Due to rounding errors in practice you will not get the solutionafter n steps.

� In practice we have n� 1. Therefore, the CG method is usedas an iterative method.

� In each iteration step there are� 1 matrix-vector product,� 2 scalar products� 3 scalar multiplications

necessary.




CG as Krylov subspace method

Theorem

The k-th iteration xk of the CG method minimizes the functionalf (·) w.r.t. the subspace

Kk(A, r0) = span{r0,Ar0,A2r0, . . . ,Ak−1r0},

i.e. there holds

f (xk) = minci

f (x0 +k−1∑i=0

ciAi r0).

The subspace Kk(A, r0) is called the k-th Krylov subspace.




CG method: Error estimate

� The error ek := xk − x∗ is measured in the energy norm

‖x‖A := (xTAx)1/2

We get the error estimate

Theorem

‖xk − x∗‖A ≤ 2

(√κ(A)− 1√κ(A) + 1

)k

‖x0 − x∗‖A

with κ(A) := cond2(A) =λmax(A)

λmin(A)≥ 1.




CG method: Example

� Consider Ax = b with

A =

(λ1 00 λ2

), b =

(λ1

λ2

).

� Use start vector x0 = (−9, 1)T . The solution is (1, 1)T .

� Looking for ε = ‖rk‖ < 10−4:




BiCG solverBi-Conjugated Gradient method

� The BiCG method belongs to the class of Krylov subspacemethods.

� It represents an further development of the CG method.

� Results from the simultaneous evaluation of Ax = b and ATx =b

� In the subsequent tutorial we extend the BiCG method toBiCGstab

� Due to this intention we examine the implementation of theBiCG method in OpenFOAM.




BiCG solverPseudocode BiCG method

Choose ψ0 ∈ Rn and ε > 0r0 = p0 := b− Aψ0, w0 := PLr0, r

?0 = p?0 := b− ATψ0, w

?0 := PLr

?0

r1 = r0, r?1 = r?0 , j := 1

While ‖rj‖2 > ε

wj := PLrj , w?j := PLr

?j

βj :=(wj , r?j )2

(wj−1, r?j−1)2

pj := wj + βjpj−1, p?j := w?j + βjp

?j−1

αj :=(wj , r?j )2

(Apj ,p?j )2

rj+1 := rj − αjApj , r?j+1 := r?j − αjATp?j

ψj := ψj−1 + αjpjj := j + 1




PBiCG solver codePBICG.C

� The PBiCG method in OpenFOAM is a modified version. Theorder of calculation has been inverted.

� ψn in the code equates to the iteration vector xn// --- Precondition residuals

preconPtr-precondition(wA, rA, cmpt);

preconPtr->preconditionT(wT, rT, cmpt);

...

// --- Update search directions:

wArT = gSumProd(wA, rT);

scalar beta = wArT/wArTold;

wj := PLrj

w?j := PLr?j

βj :=(wj , r

?j )2

(wj−1, r?j−1)2




PBiCG solver codePBICG.C

for (register label cell=0;

cell<nCells; cell++)

{

pAPtr[cell] = wAPtr[cell]

+ beta*pAPtr[cell];

pTPtr[cell] = wTPtr[cell]

+ beta*pTPtr[cell];

}

// --- Update preconditioned residuals

matrix_.Amul(wA,pA interfaceBouCoeffs_,

interfaces_, cmpt);

matrix_.Tmul(wT,pT interfaceIntCoeffs_,

interfaces_, cmpt);

scalar wApT = gSumProd(wA, pT);

...

// --Update solution and residual:

scalar alpha = wArT/wApT;

pj := wj + βjpj−1,

p?j := w?j + βjp?j−1

αj :=(wj , r

?j )2

(Apj ,p?j )2




PBiCG solver code (cont.)PBICG.C

for (register label cell=0;

cell<nCells; cell++)

{

psiPtr[cell]+= alpha*pAPtr[cell];

rAPtr[cell] -= alpha*wAPtr[cell];

rTPtr[cell] -= alpha*wTPtr[cell];

}

ψj := ψj−1 + αjpj

rj+1 := rj − αjApj

r?j+1 := r?j − αjATp?j




BiCG method

� For the BiCG method, orthogonality is assured by the Petrov-Galerkin condition i.e.

� r0 residual vector with r0 := b− Ax0

� xm ∈ x0 + Km and

Lm = KTm = span{r0,A

T r0, . . . , (AT )m−1r0}

Disadvantages BICG method� There are multiplications with AT needed.� For a regular matrix, the method could terminate without solu-

tion� The method has no minimization properties for the iteration

vector. This could cause an oscillatory behaviour in the conver-gence




Tutorial PBiCGstab

Motivation

� We have just seen that the method has some disadvantages.� We consider the BiCGstab method from van der Vorst [8] which

avoids these disadvantages.� In general, BiCGstab has smoother convergence properties� Multiplication with AT is not necessary.

� Furthermore, we want to know how to expand a linear systemsolver in OpenFOAM.




Tutorial PBiCGstabGoal

Goal

� Extension of the BiCG solver (PBiCG.C) to the stabilized versionBiCGStab

Proceeding

� Change to the directory $WM_PROJECT_USER_DIR$

� Clone the folder PBICGcp -r $FOAM_SRC/OpenFOAM/matrices/lduMatrice/solver/PBICG .

mv PBiCG PBiCGSTAB

cd PBiCGSTAB

� Change all names, pre- & post-fixes in PBiCGSTAB.C/.H: stringPBICGSTAB ⇒ PBICGSTAB:

rename ’s/PBiCG/PBiCGSTAB/g’ *

sed -i ’s/PBiCG/PBiCGSTAB/g’ *




Tutorial BiCGstabPseudo-code BiCGstab (from: Meister [9])

Choose x0 ∈ Rn und ε > 0,

r0 = p0 := b− Ax0, rP0 = pP0 := PLr0, ρP0 := (rP0 , r

P0 )2, j := 0

While ‖rj‖2 > ε

vj := ApPj , vPj := PLvj

αPj :=

ρPj(vPj , rP0 )2

, sj := rj − αPj vj , sPj := PLsj

tj := AsPj , tPj := PLtj

ωPj :=

(tPj , sPj )2

(tPj , tPj )2

xPj+1 := xPj + αPj p

Pj + ωP

j sPj

rj+1 := sj − ωPj tj , rPj+1 := sPj − ωP

j tPj

ρPj+1 := (rPj+1, rP0 )2, β

Pj :=

αPj

ωPj

ρPj+1

ρPjpPj+1 := rPj+1 + βP

j (pPj − ω

Pj v

Pj ), j := j + 1

xj = PRxPj




Tutorial BiCGstab

Procedure

� Translate the pseudo-code of the previous slide into source code

� You only have to change the file PBiCGSTAB.C

� Use the code from PBiCG.C and his pseudo-code as pattern

� If you get in trouble look after the example code BiCGSTAB.C

� Try to track the algorithm in BiCGSTAB.C

� In OpenFOAM code:� x = psi� ρ = alpha1� r0 = r0� rP = rP

. . .




Tutorial BiCGstabMake

� Add the following entries to your Make files� files

./PBiCGSTAB.C

LIB = \$(FOAM_USER_LIBBIN)/PBiCGSTAB

� options

EXE_INC = -I$(OBJECTS_DIR)

LIB_LIBS = \

$(FOAM_LIBBIN)/libOSspecific.o \

-L$(FOAM_LIBBIN)/dummy -lPstream \

-lz

� Or add the entries to the Make file for our common user-definedlibrary libmylib




Tutorial BiCGstabPBiCGSTAB.C -- ::solve I

// * * * * * * * * * * * * * * * Member Functions * * * * * * * * * * * * * //

Foam::lduMatrix::solverPerformance Foam::PBiCGSTAB::solve

(

scalarField& psi,

const scalarField& source,

const direction cmpt

) const

{

// --- Setup class containing solver performance data

lduMatrix::solverPerformance solverPerf

(

lduMatrix::preconditioner::getName(controlDict_) + typeName,

fieldName_

);

register label nCells = psi.size();

scalar* __restrict__ psiPtr = psi.begin();

scalarField pA(nCells);

scalar* __restrict__ pAPtr = pA.begin();

scalarField wA(nCells);

scalar* __restrict__ wAPtr = wA.begin();

scalarField vA(nCells);

scalar* __restrict__ vAPtr = vA.begin();

scalarField sA(nCells);

scalar* __restrict__ sAPtr = sA.begin();

scalarField tA(nCells);

scalar* __restrict__ tAPtr = tA.begin();

// --- Calculate A.psi

matrix_.Amul(wA, psi, interfaceBouCoeffs_, interfaces_, cmpt);




Tutorial BiCGstabPBiCGSTAB.C -- ::solve II

// --- Calculate initial residual

scalarField rA(source - wA);

scalar* __restrict__ rAPtr = rA.begin();

scalarField rP(nCells);

scalar* __restrict__ rPPtr = rP.begin();

scalarField r0(nCells);

scalar* __restrict__ r0Ptr = r0.begin();

// --- Define help-fields

scalarField vP(nCells);

scalar* __restrict__ vPPtr = vP.begin();

scalarField sP(nCells);

scalar* __restrict__ sPPtr = sP.begin();

scalarField tP(nCells);

scalar* __restrict__ tPPtr = tP.begin();

// --- Calculate normalisation factor

scalar normFactor = this->normFactor(psi, source, wA, pA);

if (lduMatrix::debug >= 2)

{

Info<< " Normalisation factor = " << normFactor << endl;

}

// --- Calculate normalised residual norm

solverPerf.initialResidual() = gSumMag(rA)/normFactor;

solverPerf.finalResidual() = solverPerf.initialResidual();




Tutorial BiCGstabPBiCGSTAB.C -- ::solve III

// --- Check convergence, solve if not converged

if (!solverPerf.checkConvergence(tolerance_, relTol_))

{

// --- Select and construct the preconditioner

autoPtr<lduMatrix::preconditioner> preconPtr =

lduMatrix::preconditioner::New

(

*this,

controlDict_

);

// --- precondition r0

preconPtr->precondition(rP, rA, cmpt);

// --- initialize "pA and r0" with the preconditioned r0

for (register label cell=0; cell<nCells; cell++)

{

pAPtr[cell] = rPPtr[cell];

r0Ptr[cell] = rPPtr[cell];

}

// --- initialize rPr0

scalar rPr0 = gSumProd(rP,r0);

// --- Solver iteration

do

{

// --- Calculate v=A*p and precondition v

matrix_.Amul(vA, pA, interfaceBouCoeffs_, interfaces_, cmpt);

preconPtr->precondition(vP, vA, cmpt);




Tutorial BiCGstabPBiCGSTAB.C -- ::solve IV

// --- Calculate alpha=rPr0/vPr0

scalar alpha2 = gSumProd(vP, r0);

scalar alpha = rPr0/alpha2;

// --- Calculate s=rA-alpha*v


{

sAPtr[cell] = rAPtr[cell] - alpha*vAPtr[cell];

}

// --- Precondition s

preconPtr->precondition(sP, sA, cmpt);

// --- Calculate t=A*sP and precondition t

matrix_.Amul(tA, sP, interfaceBouCoeffs_, interfaces_, cmpt);

preconPtr->precondition(tP, tA, cmpt);

// --- Calculate alpha1=tPsP/tPtP

scalar tPsP=gSumProd(tP,sP);

scalar tPtP = gSumProd(tP,tP);

if (solverPerf.checkSingularity(mag(tPtP)/normFactor)) break;

scalar alpha1 = tPsP/tPtP;

// --- Update solution and residual


{

psiPtr[cell] += alpha*pAPtr[cell] + alpha1*sPPtr[cell];

rAPtr[cell] = sAPtr[cell] - alpha1*tAPtr[cell];

rPPtr[cell] = sPPtr[cell] - alpha1*tPPtr[cell];

}




Tutorial BiCGstabPBiCGSTAB.C -- ::solve V

solverPerf.finalResidual() = gSumMag(rA)/normFactor;

// --- Update rPr0 and calculate beta

scalar rPr0old = rPr0;

rPr0 = gSumProd(rP, r0);

scalar beta = (alpha*rPr0)/(alpha1*rPr0old);

// --- Update search directions


{

pAPtr[cell] = rPPtr[cell] + beta*(pAPtr[cell] - alpha1*vPPtr[cell]);

}

} while

(

solverPerf.nIterations()++ < maxIter_

&& !(solverPerf.checkConvergence(tolerance_, relTol_))

);

}

return solverPerf;

}

// ************************************************************************* //




Tutorial BiCGstabTest case pitzDaily

� Solver simpleFoam

� Settings

"U|k|epsilon|R"

{

solver PBiCGSTAB;

preconditioner DILU;

tolerance 1e-05;

relTol 0.1;

}

� BiCG analogue

� Convergence after

� BiCG 952 Iterations

� BiCGSTAB 754 Iterations




Tutorial BiCGstabComparison BiCG/BiCGSTAB – Residuals

Comparison of the residuals BiCG/BiCGSTAB




Literature I

[1] Hirt, C. W., Nichols, B. D.Volume of Fluid (VOF) Method for the Dynamic sof Free Boundaries, Journal ofComputational Physics, 39: 201-225-65, 1981.

[2] Ubbink, O, Numerical prediction of two fluid systems with sharp interfaces, Ph.D Thesis, Imperial College ofScience, Technology and Medicine, London, 1997.

[3] Rusche, H. Computational Fluid Dynamics of Dispersed Two-Phase Flows at High Phase Fractions, Ph.D Thesis,Imperial College of Science, Technology and Medicine, London, 2002.

[4] Damian, S. M. Description and utilization of interFoam multiphase solver, -Final Work-Computational FluidDynamics. http://infofich.unl.edu.ar/upload/5e6dfd7ff282e2deabe7447979a16d49b0b1b675.pdf

[5] Eslamdoost, A. Forced Roll Motion of a 2D Box and Interaction with Free-Surface, PhD course in CFD withOpenSource software, Chalmers University, 2009 http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2009/

ArashEslamdoost/RollMotionofaBoxandInteractionwithFreeSurface.pdf

[6] Issa, R. I. Solution of the implicitly discretised fluid flow equations by operator splitting, Journal of ComputationalPhysics, 62: 40-65, 1985.

[7] Van Leer, B. Towards the ultimate conservative difference scheme III. Upstream-centered finite-differenceschemes for ideal compressible flow, J. Comp. Phys. 23 (3): 263–275, 1977.

[8] van der Vorst, H. A. BI-CGSTAB: A fast and smoothly converging variant of BI-CG for the solution of nonsym-metric linear systems, SIAM J. Sci. Stat. Comput., 13: 631-644, 1992.

[9] Meister, A. Numerik linearer Gleichungssysteme, Vieweg Verlag, 2011.


http://infofich.unl.edu.ar/upload/5e6dfd7ff282e2deabe7447979a16d49b0b1b675.pdf

http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2009/ArashEslamdoost/RollMotionofaBoxandInteractionwithFreeSurface.pdf

http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2009/ArashEslamdoost/RollMotionofaBoxandInteractionwithFreeSurface.pdf


Date post:	02-Aug-2020
Category:	Documents
Upload:	others
View:	13 times
Download:	0 times

Linear solvers & preconditioners · Mesh & matrixSparse adressing...

Documents