7/29/2019 Richard Guenther-Advanced Cpp

1/23

Advanced C++ Techniques inComputational Fluid Dynamics

Richard Gunther

Inst. fur Astronomie und Astrophysik

Abt. Computational Physics

p.1/23

7/29/2019 Richard Guenther-Advanced Cpp

2/23

Contents

1. Background2. Advanced Object Oriented programming with C++

C++ for Experts in 3 Minutes

Template Metaprogramming

Expression Templates

3. The POOMA Library4. Application

5. Conclusion

p.2/23

7/29/2019 Richard Guenther-Advanced Cpp

3/23

Background

Work sponsored by SFB 382, Verfahren und Algorithmenzur Simulation physikalischer Prozesse aufHchstleistungsrechnern;project C19, Dreidimensionale MRHD.Available and used codes:

RH2D, twodimensional RHD

TRAMP, threedimensional RHD

NIRVANA v0, threedimensional MHD

Goals for C19:

Modern (dont use Fortran 77)

Parallel (MPI/OpenMP)

Full-Featured (physically and technically) p.3/23

7/29/2019 Richard Guenther-Advanced Cpp

4/23

Object Oriented programming with C++

OO in C++ can be done using the following well-knownabstraction mechanisms:

Classes

Class inheritance

Virtual methods

Operator overloadingAnd other, less well-known:

Multiple (virtual) class inheritance

Templates and (partial) specialization

p.4/23

7/29/2019 Richard Guenther-Advanced Cpp

5/23

C++ features suitable for CFD

Avoid virtual methods and multiple inheritance inperformance critical sections.

Use traits classes and (partial) specialization to ease

maintainance and avoid runtime conditionals.

Use operator / function overloading for complex typesto allow rapid development and ease maintainance.

Operator overloading may expose costly temporaries forfat objects. Using template metaprogramming can solve

this.Veldhuizen, 2000 (Techniques for Scientific C++)

p.5/23

7/29/2019 Richard Guenther-Advanced Cpp

6/23

Template Specialization

Template specialization and function overloading as SwissArmy Knife!

template class A;

class B;template class A

{

void foo();};

template class A

{void bar();

};

p.6/23

7/29/2019 Richard Guenther-Advanced Cpp

7/23

Function Overloading

Partial specialization of function templates not allowed.Do the trick using function overloading:

template struct WrappedInt {};

template void doSomething(Op&, Domain&, WrappedInt);

template

void doSomething(Op&, Domain&, WrappedInt);

p.7/23

7/29/2019 Richard Guenther-Advanced Cpp

8/23

Partial Template Specialization

or using partial template specialization of a wrapper class:

template

7/29/2019 Richard Guenther-Advanced Cpp

9/23

Template Metaprogramming, I

Let the compiler do static parts of algorithms!

Possible with templates and specialization

Template specialization makes up a Turing-Complete

language.

template struct Factorial {

static const int result

= Factorial::result * n;

};

template struct Factorial {static const int result = 1;

};

const int factorial_of_7 = Factorial::result;

p.9/23

T l M i II

7/29/2019 Richard Guenther-Advanced Cpp

10/23

Template Metaprogramming, II

Template metaprogramming is Lisp-like:

Thinking the recursive way

Using recursive data-structures like lists and trees

Building static information into types

Lots of ugly templates

You pay for it through compile time regressions.

p.10/23

E i T l t I

7/29/2019 Richard Guenther-Advanced Cpp

11/23

Expression Templates, I.

Avoid temporaries in evaluating expressions for fat objectslike Vectors, Tensors or complete Arrays

Allow generic functions to operate on expressions likeusual objects

Possible using Expression Templates (Veldhuizen, 1995)!

p.11/23

E i T l t II

7/29/2019 Richard Guenther-Advanced Cpp

12/23

Expression Templates, IIa.

1.0 x

+

/

x BinaryNodeReference

ReferenceScalar

Expr

The complete type representing the expression would

look likeExpr

7/29/2019 Richard Guenther-Advanced Cpp

13/23

Expression Templates, IIb.

The operator= of the type T triggers evaluation of theexpression

template T& T::operator=(Expr&);

And finally creates an instance of the overloadedoperator= which looks like (for vector-like T)

for (int i=0; i

7/29/2019 Richard Guenther-Advanced Cpp

14/23

Expression Templates, III

Use computations on the expression type to selectalgorithms to query

Patch intersections

Used guards

Necessary communication

Straightforward to allow serial, MPI and OpenMP imple-mentation to co-exist.

p.14/23

The POOMA Library

7/29/2019 Richard Guenther-Advanced Cpp

15/23

The POOMA Library

The POOMA library was created at ACL/LANL (Reynderset al., 1996), now free software maintained atCodeSourcery (www.pooma.com).Extensively uses expression templates through the PETElibrary. Provides a framework for parallel grid-based CFD.

Arrays, Fields

Tiny objects like Vectors, Tensors, Matrices

Fields contain

Multiple components and centering points

Mesh abstraction

Relations between Fields and for BCs

p.15/23

Extending POOMA

7/29/2019 Richard Guenther-Advanced Cpp

16/23

Extending POOMA

To suite our needs POOMA was extended by

Non-uniform meshes and spherical and cylindricalcoordinates

HDF5 serial and parallel I/O

Native OpenMP parallelization

Native MPI parallelization

Glue to PETSc library

Autogeneratable reference documentation

p.16/23

Using POOMA

7/29/2019 Richard Guenther-Advanced Cpp

17/23

Using POOMA

Program finite differences using and combining any of

Expression templates

Simple stencils

Generic stencil-like sub-programs

Patch-local sub-programs

p.17/23

Using POOMA Expression templates

7/29/2019 Richard Guenther-Advanced Cpp

18/23

Using POOMA, Expression templates

Using expression templates is as simple as writingexpressions with Arrays or Fields as arguments:

rho = rho0 * pow(positions(rh).comp(0), -1.5)

* exp(-pow(cos(positions(rh).comp(1))/0.05, 2));

( ! "#

$

%

'

!

(

#

) 12

3

4

5

47

8

)Expression template support in POOMA includes

reductions and selections:rho = where(rho < 1e-6, 1e-6);

dt = min(norm(spacings(v)/(v+cs)));

p.18/23

Using POOMA simple stencils

7/29/2019 Richard Guenther-Advanced Cpp

19/23

Using POOMA, simple stencils

Finite differencing of one quantity

struct Gradient {

template

double operator()(const F& f, int i, int j)

{

return 0.5*(f.read(i+1,j) - f.read(i-1,j))

+ 0.5*(f.read(i,j+1) - f.read(i,j-1));}

};

flux = Stencil()(rho*v);

rho = dt * flux;

p.19/23

Using POOMA stencil-like sub-programs (a)

7/29/2019 Richard Guenther-Advanced Cpp

20/23

Using POOMA, stencil like sub programs (a)

Complex local operations

struct MinMod {

template

void operator()(LHS& l, RHS1& r1, RHS2& r2,

Loc& I)

{

if (r1(I) > 0 && r2(I) > 0)l(I) = std::min(r1(I), r2(I));

else if (r1(I) < 0 && r2(I) < 0)

l(I) = std::max(r1(I), r2(I));else

l(I) = 0;

}};

p.20/23

Using POOMA stencil-like sub-programs (b)

7/29/2019 Richard Guenther-Advanced Cpp

21/23

Using POOMA, stencil like sub programs (b)

ScalarCode()(s,

Stencil(rho*v),

Stencil(rho*v),

Stencil(rho*v));

Or use manual written loops per patch, like for glueing toFortran code.

Its a trade-off between re-usable components and classicstyle of writing code.

p.21/23

Application

7/29/2019 Richard Guenther-Advanced Cpp

22/23

Application

Using all this for 3d MRHD. Current state:

threedimensional parallel ideal hydro in spherical,cylindrical and Cartesian coordinates

Poisson solver for self-gravity in Cartesian coordinates

Physical applications

Stellar oscillations as a test-problem

3d non-viscous protoplanetary accretion disks

p.22/23

Conclusion

7/29/2019 Richard Guenther-Advanced Cpp

23/23

Conclusion

Using C++ for Computational Fluid Dynamics applicationsis feasible.

Available libraries for advanced C++ like Boost

Available libraries for (parallel) arrays like POOMA,Blitz++

As easy as using Fortran9x

Performance comparable to C, Fortran

Free compilers available!

More people know C++ than Fortran9x

p.23/23