Development of spectral element code for incompressible ... · Development of spectral element code...

Hungarian Academy of Sciences Hungarian Academy of Sciences KFKI Atomic Energy Research InstituteKFKI Atomic Energy Research Institute

Development of spectral element code for incompressible fluid

flows

PPééterter KKáávrvráánn

16th AER Symposium16th AER SymposiumBratislava, SlovakiaBratislava, Slovakia

2525--29 29 SpetemberSpetember 20062006

22


OverviewOverviewThe aim of the project,The aim of the project,Spectral element method,Spectral element method,NavierNavier--Stokes solver,Stokes solver,Implementation issues,Implementation issues,ResultsResults

33


The aim of the projectThe aim of the project

To develop an accurate To develop an accurate NavierNavier--Stokes solver, which isStokes solver, which is

based on spectral element method,based on spectral element method,

applicable to arbitrary 2D and 3D geometry,applicable to arbitrary 2D and 3D geometry,

arbitrary arbitrary DirichletDirichlet and Neumann boundary conditions can be and Neumann boundary conditions can be implemented into it ,implemented into it ,

applicable largeapplicable large--scale problemsscale problems

44


FiniteElementMethod

SpectralMethod

SpectralElementMethod

Spectral Element MethodSpectral Element MethodSpectral element method (SEM) is a subset of finite element methSpectral element method (SEM) is a subset of finite element methods, and ods, and

a a multidomainmultidomain spectral method,spectral method,

∑∑= =

=N

m

N

n

Nn

Nm

emn

eN llu),(u

1 1)()( ηξηξ

DifferencesDifferences between between SEMSEM and traditional and traditional FEMFEM::

Traditional Traditional FEMFEM uses lowuses low--order interpolation order interpolation functions,functions,

SEMSEM uses highuses high--order order LagrangianLagrangian interpolation interpolation functions based on functions based on JacobiJacobi--polinomialspolinomials::

basis Lobatto-Legendre-Gauss))(()1(

)()1()(2

mmN

NNm LNN

Llξξξ

ξξξ−+

′−−=

2D basis function, N=10

55


in higher dimensions bases are the tensorin higher dimensions bases are the tensor--product of the oneproduct of the one--dimensional basesdimensional bases

in 2Din 2D--> rectangular, in 3D > rectangular, in 3D --> > cuboidcuboid elements,elements,

computations are made on reference computations are made on reference elementselements

reference elements are mapped to the reference elements are mapped to the deformed elements using deformed elements using isoparametricisoparametricmapping.mapping.

good skew mesh tolerance, good skew mesh tolerance,

ξ

-1 0 1-1

0

1η

2D reference element2D reference element

GaussGauss--LegendreLegendre--LobattoLobatto pointspoints

66


Some propertiesSome properties: :

Operator Costs (3D):Operator Costs (3D):

Standard FEMStandard FEM** SEMSEM# memory accesses: O( EN 6 ) O( EN 3 )# operations: O( EN 6 ) O( EN 4 )

N = order, E = number of elements, EN 3 = number of gridpoints

handles handles complex geometrycomplex geometry, ,

Spatial Spatial discretizationdiscretization error decays exponentiallyerror decays exponentially with the order of the with the order of the LagrangianLagrangianbases,bases,matrixmatrix--vector products: vector products:

⎯⎯ discretizeddiscretized operatorsoperators in spectral element method represented by in spectral element method represented by tensortensor--product matricesproduct matrices

⎯⎯ matrixmatrix--vector products cast as efficient matrixvector products cast as efficient matrix--matrix products:matrix products:

Ty

x

UDuIDuD

DUuDIuD

=⊗=

=⊗=

)(

)(Elemental derivative operator:

77


Spectral element method for incompressible fluid flowsSpectral element method for incompressible fluid flows• TimeTime--dependentdependent NavierNavier--Stokes equation:Stokes equation:

•• use of Puse of PNN--PPNN--22 methodmethod:

P=5 element gridP=5 element grid

⎯⎯for theoretical reasons use staggered grid to avoid for theoretical reasons use staggered grid to avoid unphysical solutionsunphysical solutions

––velocity is approximated by P=velocity is approximated by P=NN order Gaussorder Gauss--LegendreLegendre--LobattoLobatto polinomialspolinomials,,

– pressurepressure is approximated by P=is approximated by P=NN--2 2 order Gaussorder Gauss--LegendreLegendre polinomialspolinomials.

⎪⎭

⎪⎬

⎫

=

++⋅−=+∂

∂

0

1)(

Du

MfDLuuuCu

M pTt ρ

ν≡=⋅∇

+∇−Δ=∇⋅+∂∂

⎪⎭

⎪⎬

⎫

0u

uuuu fpt ρ

ν 1

88


Temporal Temporal discretizationdiscretization::

Temporal Temporal discretizationdiscretization is based on Operatoris based on Operator--IntegrationIntegration--Factor Splitting Factor Splitting Method (OIF),Method (OIF),

the main idea is to split operators and integrate them separatethe main idea is to split operators and integrate them separately,ly,

convection termconvection term expressed as material derivative, and expressed as material derivative, and discretizeddiscretized by 2nd by 2nd order backward differencing scheme:order backward differencing scheme:

)~(2

~3~4~ 12n

nnnS

tuuuu

=Δ

+− −− qn−u~ -- pure convection term computedpure convection term computed inin

by 4thby 4th--RungeRunge--KuttaKutta],[ nqn tt −

The The discretizeddiscretized unstable Stokes unstable Stokes --systemsystem to be solved implicitly:to be solved implicitly:

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟

⎟⎠

⎞⎜⎜⎝

⎛

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−

0

nnT Bfu0DDH

np

)~( nS u -- Stokes operatorStokes operator

.23

0 ,0 =Δ+−= ⎟⎠⎞

⎜⎝⎛ β

βν MLH t -- HelmholtzHelmholtz--matrixmatrix

D , DT -- divergence, gradient operatordivergence, gradient operator

99


After applying the splitting method to the system After applying the splitting method to the system (according to(according to Paul Fischer: An Paul Fischer: An overlapping Schwarz method for spectral element solution of overlapping Schwarz method for spectral element solution of NavierNavier--Stokes equations, Stokes equations, J. of Comp. J. of Comp. Phys. Phys. 133,133, 8484--101), 101),

The algorithm to solve the StokesThe algorithm to solve the Stokes--problem (for a problem (for a dd--dimensional system):dimensional system):

nnt ppp −== +−Δ 1 ,0

δβT1DDME

1n1n MfDuH ++ += npT~

1nuDE +−= ~pδ

1nT11n uDMu +−Δ+ += ~0

pt δβ

ppp nn δ+=+1

1.1.

2.2.

3.3.

4.4.

dd HelmholtzHelmholtz--systemsystem

11 Pressure correction stepPressure correction step

The The d+1d+1 equations are solved by equations are solved by Preconditioned Conjugate Gradient Preconditioned Conjugate Gradient methodmethod

1010


Implementation in C++ language,Implementation in C++ language,

objectobject--oriented designoriented design::

defining highdefining high--level data types, that represents geometrical, and algebraic level data types, that represents geometrical, and algebraic objects,objects,

operationsoperations are evaluated on are evaluated on abstract mathematical objectsabstract mathematical objects,,

the geometry and the linear algebra are decoupled,the geometry and the linear algebra are decoupled,

linear solverslinear solvers are are geometrygeometry (and problem) (and problem) independentindependent, ,

Example: Example: LinearSolver::PCG(MatrixSystem* mat_sys, Vector* lhs, Vector* rhs, Vector* BC);

use of standard libraries in mathematical modulesuse of standard libraries in mathematical modules::

-- BLAS (Basic Linear Algebra Subroutines)BLAS (Basic Linear Algebra Subroutines)

-- PETScPETSc (Portable Extensible Toolkit for Scientific Computing) (Portable Extensible Toolkit for Scientific Computing)

Code ImplementationCode Implementation

1111


Geometry GenerationGeometry Generation

treat arbitrary 2D and 3D geometries, and treat arbitrary 2D and 3D geometries, and

approximate curved geometry as good as possibleapproximate curved geometry as good as possible

mesh generation must be usermesh generation must be user--friendly friendly

The aimThe aim::

Realization:Realization:

Coarse finite element mesh is generated by Coarse finite element mesh is generated by GAMBIT GAMBIT meshmesh--generation program,generation program,

the code can handle 4the code can handle 4-- and 9and 9--node quadrilateral, and 8node quadrilateral, and 8-- and 27and 27--node hexahedral node hexahedral elements elements

SEM SEM intrinsicallyintrinsically uses quadrilateral (2D) and hexahedral (3D) elementsuses quadrilateral (2D) and hexahedral (3D) elements

code reads GAMBIT files in Neutral File Format (.code reads GAMBIT files in Neutral File Format (.neuneu) )

spectral element mesh is built on the coarse finite element meshspectral element mesh is built on the coarse finite element mesh

1212


Parallel ImplementationParallel Implementationparallel implementation is based on parallel implementation is based on Domain Decomposition Method,Domain Decomposition Method,

divide the computational domain into subdivide the computational domain into sub--domains, containing a branch of elements,domains, containing a branch of elements,

SubSub--domain solverdomain solver

Process #4


Process #3


Process #2

Communication on the Communication on the boundaries of subboundaries of sub--domains,domains,

CommunicationCommunication


Process #1

Automatic mesh partitioning with Automatic mesh partitioning with MetisMetis library routines (n process library routines (n process --> n partition),> n partition),

parallel communication is realized by MPI and parallel communication is realized by MPI and PETScPETSc librarieslibraries

Single Program Multiple Data approach,Single Program Multiple Data approach,

1313


Basic FlowchartBasic Flowchart

FE mesh generationFE mesh generationGAMBITGAMBIT mesh generation, using 4mesh generation, using 4--, 9, 9--node quadrilateral, 8node quadrilateral, 8--, ,

2727--node hexahedral elementsnode hexahedral elementsset boundary condition typesset boundary condition typesreads reads GAMBIT neutral (.GAMBIT neutral (.neuneu) files) files

FE mesh partitioningFE mesh partitioningMesh partitioning (Mesh partitioning (MetisMetis),),number of partitions = number of number of partitions = number of processesprocessesdistribute distribute submeshessubmeshes among processesamong processes

Build SE meshBuild SE mesheach process builds a spectral element mesh on its FE each process builds a spectral element mesh on its FE

submeshsubmesh,,create parallel communication patterns, create parallel communication patterns,

Parallel CalculationsParallel Calculationsdefine boundary conditions, define boundary conditions, build discrete operators,build discrete operators,perform parallel calculations, perform parallel calculations, save resultssave results

1414


Test casesTest casesKovasznayKovasznay flowflow

Analytical solution of 2D steadyAnalytical solution of 2D steady--state state NavierNavier--Stokes equation,Stokes equation,incorporates nonlinear terms, good test for the full incorporates nonlinear terms, good test for the full NavierNavier--Stokes equationStokes equationCheck spectral convergence of the SEMCheck spectral convergence of the SEM

22

2

44

Re2

Re

2/)1(

2/2sin

2cos1

πλ

ππλ

π

λ

λ

λ

−−=

−=

=

−=

x

x

x

ep

yev

yeu

relδmax

N - polinomial order

Spectral element convergence of velocitySpectral element convergence of velocity

1515


2D lid2D lid--driven cavitydriven cavity

2D cavity flow, 10x10, N=10 order elements, vorticity contours (left), pressure contours (center), velocity contours (right) at t=12. Re=1000.

V=1V=1

1616


Flow directionFlow direction

SubchannelSubchannel

Direct Numerical Simulation of flows in a Direct Numerical Simulation of flows in a subchannelsubchannel of a rod bundleof a rod bundleComputational problem definition:Computational problem definition:

diameter of the rod= 9.1mmdiameter of the rod= 9.1mmdistance between rods=12.2mmdistance between rods=12.2mmlength of the computational domain = 10mmlength of the computational domain = 10mmeach sides of the hexagon is coupled periodicallyeach sides of the hexagon is coupled periodicallynono--slip wall boundary condition on the surface of the slip wall boundary condition on the surface of the

rod rod

triangular array of fuel rods triangular array of fuel rods

1717


Coarse GAMBIT mesh of the Coarse GAMBIT mesh of the subchannelsubchannel

GAMBIT mesh: 27GAMBIT mesh: 27--node hexahedral node hexahedral elements in order to follow curved elements in order to follow curved geometry of the rodgeometry of the rod

10 layers, 72 elements in each layer 10 layers, 72 elements in each layer = 720 element= 720 element

SE mesh: expansion order N=12,SE mesh: expansion order N=12,

~ 1.2 * 10~ 1.2 * 1066 velocity nodes, ~ 9.5 * velocity nodes, ~ 9.5 * 10105 5 pressure nodespressure nodes

elements are concentrated at the elements are concentrated at the wall in order to resolve boundary layerwall in order to resolve boundary layer

1818


Laminar flow around in a Laminar flow around in a subchannelsubchannel

Computed solution Computed solution Relative difference between approximate Relative difference between approximate analytical and computed results analytical and computed results

m/sm/s

relative difference is independent of expansion orderrelative difference is independent of expansion order

There is an approximate analytical solution for stationary lamiThere is an approximate analytical solution for stationary laminar flow in nar flow in triangular array of rodstriangular array of rods

1919


Towards turbulent flow modelingTowards turbulent flow modelingDirect numerical simulation (DNS) solves Direct numerical simulation (DNS) solves NavierNavier--Stokes equations as accurate as possible Stokes equations as accurate as possible

without turbulence models,without turbulence models,

There are attempts for DNS simulation of turbulent flows in roThere are attempts for DNS simulation of turbulent flows in rod bundles:d bundles:

Paul Fischer uses SEM for DNS of Paul Fischer uses SEM for DNS of ReRe=30.000 flow in rectangular array,=30.000 flow in rectangular array,

Mayer Mayer GusztGusztáávv uses lattice uses lattice BoltzmannBoltzmann method for DNS and LES in triangular array,method for DNS and LES in triangular array,

DNS of turbulent flows in rod bundle helps us to understand mixDNS of turbulent flows in rod bundle helps us to understand mixing processes:ing processes:

Projection of mean lateral velocity to a cross-section in LES simulation using lattice Boltzmannmethod.

2020


Numerical approachNumerical approach

common practice in periodic channel flow simulation is to use ccommon practice in periodic channel flow simulation is to use constantonstant--body body force term to accelerate the flow in flow direction,force term to accelerate the flow in flow direction,

our approach is to adjust forcing term to maintain constant fluour approach is to adjust forcing term to maintain constant flux in the axial x in the axial direction,direction,

tests are run with different expansion order and different meshetests are run with different expansion order and different meshes,s,

no statistical quantities are available yet,no statistical quantities are available yet,

flow of flow of ReRe=5000=5000

2121


m/s

Contours of velocity component in flow direction in a vertical cross-section.

Flow direction

m/s

Snapshot of the velocity field in the subchannel

2222


Contours of velocity component in Contours of velocity component in flow direction, in a cross sectionflow direction, in a cross section

Lateral velocity vectors in a cross Lateral velocity vectors in a cross sectionsection

m/s

2323


ConclusionsConclusions

A simulation code for incompressible fluid flows has been develA simulation code for incompressible fluid flows has been developed,oped,

based on spectral element method,based on spectral element method,

capable of handling arbitrary complex geometries,capable of handling arbitrary complex geometries,

can be run on parallel, distributed memory architecture (computcan be run on parallel, distributed memory architecture (computerer--cluster),cluster),

implemented in c++, with object oriented design (~50000 lines oimplemented in c++, with object oriented design (~50000 lines of code)f code)

several test cases have been run,several test cases have been run,

tests of turbulent flows are runningtests of turbulent flows are running

Future work:Future work:

compare simulation results with available experimental data forcompare simulation results with available experimental data for flows Re~10000,flows Re~10000,

increase parallel efficiency, increase parallel efficiency,

2424


Thank you Thank you for your attentionfor your attention!!

Date post:	07-Sep-2018
Category:	Documents
Upload:	danghanh
View:	221 times
Download:	0 times

Development of spectral element code for incompressible ... · Development of spectral element code...

Documents