12 March 2004 A High-Fidelity, High-Performance Turbulent Dispersion Framework Damian Rouson...

12 March 200412 March 2004

A High-Fidelity, High-A High-Fidelity, High-Performance Turbulent Performance Turbulent Dispersion Framework Dispersion Framework

Damian RousonDamian RousonCombustion Modeling & ScalingCombustion Modeling & ScalingU.S. Naval Research Laboratory, U.S. Naval Research Laboratory,

Washington, DCWashington, DC

Prof. Joel Koplik & Karla MorrisProf. Joel Koplik & Karla MorrisDepts. of Physics & Mechanical Depts. of Physics & Mechanical

EngineeringEngineeringThe City College of CUNY, New York, NYThe City College of CUNY, New York, NY

Sponsors: NSF, NRLSponsors: NSF, NRL

OverviewOverview Motivation & ObjectivesMotivation & Objectives Previous workPrevious work PhysicsPhysics

– Fire suppressionFire suppression– Quantum turbulenceQuantum turbulence

SoftwareSoftware– Componentization strategyComponentization strategy– O.O. Fortran 90/95 ImplementationO.O. Fortran 90/95 Implementation

Future WorkFuture Work– Study Study

architecturearchitecturecomplexitycomplexityperformanperformancece

– Study fine-grained parallelism (OpenMP)Study fine-grained parallelism (OpenMP)

MotivationMotivation

Motivation Motivation ObjectivesObjectivesPrevious WorkPrevious WorkPhysicsPhysicsSoftwareSoftwareFuture WorkFuture Work

NN ObjectsObjects

00 Point masses: particles, Point masses: particles, dropletsdroplets

11 Quantum vortices, “snails”Quantum vortices, “snails”

22 Interfaces (fluid/fluid & Interfaces (fluid/fluid & fluid/solid)fluid/solid)

33 Two-fluid mixturesTwo-fluid mixtures

A broad class of phenomena involve A broad class of phenomena involve interactions between fluid turbulence & interactions between fluid turbulence & N-dimensional objectsN-dimensional objects

Algorithms & data structures for these Algorithms & data structures for these problems exhibit a high degree of problems exhibit a high degree of commonality.commonality.

Exploiting software design Exploiting software design commonalities can lead to greater code commonalities can lead to greater code reuse and more efficient interoperationreuse and more efficient interoperation

To build an extensible set of software To build an extensible set of software components suitable for the N={0,1} subset components suitable for the N={0,1} subset of turbulent dispersion problems.of turbulent dispersion problems.

To quantify the influence of object-oriented To quantify the influence of object-oriented design & componentization on software design & componentization on software complexity and performance.complexity and performance.

To simulate new physics & improve To simulate new physics & improve engineering models:engineering models:– Fire suppressionFire suppression– Quantum turbulence Quantum turbulence – Boundary-layer combustionBoundary-layer combustion

ObjectivesObjectives

MotivationMotivationObjectivesObjectivesPrevious WorkPrevious WorkPhysicsPhysicsSoftwareSoftwareResultsResultsConclusions & Future WorkConclusions & Future Work

Previous WorkPrevious Work

MotivationMotivationObjectivesObjectivesPrevious WorkPrevious WorkPhysicsPhysicsSoftwareSoftwareFuture WorkFuture Work

Rouson & Eaton (2001) “On the preferential Rouson & Eaton (2001) “On the preferential concentration of solid particles in a fully-developed concentration of solid particles in a fully-developed turbulent channel flow”, turbulent channel flow”, J. Fluid Mech.J. Fluid Mech.

Rouson & Xiong (2004) “Design metrics in quantum Rouson & Xiong (2004) “Design metrics in quantum turbulence simulations: How physics influences turbulence simulations: How physics influences software architecture”, software architecture”, Scientific ProgrammingScientific Programming..

Decyk, Norton & Szymanski (1997-) Object-oriented Decyk, Norton & Szymanski (1997-) Object-oriented programming in Fortran 90, programming in Fortran 90, ACM Fortran ForumACM Fortran Forum, , etc.etc.

Akin (2003), Akin (2003), Object-Oriented Programming via Object-Oriented Programming via Fortran 90/95Fortran 90/95, Cambridge U. Press., Cambridge U. Press.

Armstrong et al. (1999-) “Toward a common Armstrong et al. (1999-) “Toward a common component architecture for high-performance component architecture for high-performance scientific computing”scientific computing”

Particle-Laden FlowParticle-Laden FlowDirect Numerical Simulation (DNS) of Direct Numerical Simulation (DNS) of Navier-Stokes Equations:Navier-Stokes Equations:

Spectral Spatial Representation:

0/

/Re)/1(///

jj

jjiijiji

xu

xxuxpxuutu

fp

DDD

iiiDi

ii

St

C

gStuCdtd

dtdx

/

Re/Re124

/)v)((/v

v/

687.0

x

Lagrangian Particle Equation of Motion:

zyx

zyx

zyxkkk

zkykxkikkk ett

,,

)()(ˆ),( uxu

3rd-Order Runge-Kutta Time Advancement


Isotropic turbulence:

Particle-Laden Particle-Laden FlowFlow

Particles near Particles near centerplane of a centerplane of a channel flow:channel flow:

Preferential concentration: Particles move independently Particles move independently

from each other and slip from each other and slip relative to flowrelative to flow

Coherent flow structures Coherent flow structures segregate particles segregate particles according to their inertiaaccording to their inertia– e.g. “cyclone separation” by e.g. “cyclone separation” by

vortices vortices


Below 2.17 K, liquid helium behaves as a two-Below 2.17 K, liquid helium behaves as a two-fluid mixture:fluid mixture: Normal viscous fluidNormal viscous fluid Inviscid superfluidInviscid superfluid

– – characterized by discrete (quantized) characterized by discrete (quantized) vortex motionsvortex motions

– – vortex core ~ 1 A acrossvortex core ~ 1 A across

– – can be modeled by can be modeled by

1. Nonlinear Schrodginger Eq.1. Nonlinear Schrodginger Eq.

2. Euler equation 2. Euler equation

3. Discrete vortex equations3. Discrete vortex equationsSurprisingly, turbulence statisticsSurprisingly, turbulence statistics

match those of classical flows.match those of classical flows.

Quantum TurbulenceQuantum Turbulence

Density ratio of each Density ratio of each fluid:fluid:


SuperfluidSuperfluid: vortex filament method. : vortex filament method.


3

00

////

/)(4

)]([)(

rSSrSv

vvvSSvvvSvvS

d

dt

d

i

isnisnis

),( tS

Arbitrary originArbitrary origin

),(

),(/t

tS

S

Increasing Increasing

Vortex filamentVortex filament

Normal fluid: Normal fluid: DNS of Navier-Stokes DNS of Navier-Stokes equations.equations.

Material properties: Material properties: ’’Quantum of circulation:Quantum of circulation: = ħ/m= ħ/mHeHe= 9.97*10= 9.97*10-4-4 cm cm22/sec/sec

Equations of motion:Equations of motion:

(Biot-Savart Law)(Biot-Savart Law)


Animation Animation Barenghi et al. (1997)Barenghi et al. (1997) Our resultOur result

)2

cos()2

sin(

)2

cos()2

sin(

)2

cos()2

sin(

xB

yCv

zA

xBv

yC

zAv

z

y

x

A=B=C=0.748cm/s, 0.2cm

As a prelude to turbulence, we studied the As a prelude to turbulence, we studied the steady Arnold-Beltrami-Childress (ABC) steady Arnold-Beltrami-Childress (ABC) flow:flow:



ComponentizationComponentization

FluidFluidProvides: Provides: d_dt(), u(x)d_dt(), u(x)

Uses: Uses:

f(x)f(x)

VectorFielVectorFieldd

GridGrid

CloudCloudProvides: Provides: d_dt(), f(x)d_dt(), f(x)

Uses: Uses:

u(x)u(x)

DropleDroplett

TangleTangleProvides: Provides: d_dt(), f(x)d_dt(), f(x)

Uses:Uses:

u(x)u(x)

VortexPoinVortexPointt

IntegratorIntegratorProvides:Provides: eulereuler RK3RK3

Uses:Uses:

d_dt()d_dt()


OOP in Fortran 90/95OOP in Fortran 90/95

OOP OOP ConceptConcept

Fortran Fortran ConstructConstruct

ClassClass MODULEMODULE

Abstract data Abstract data typetype

Derived typeDerived type

InheritanceInheritance AggregationAggregation

PolymorphisPolymorphismm

Generic Generic interfacesinterfaces


Fortran ImplementationFortran ImplementationMODULE Cloud_ClassMODULE Cloud_Class USE Droplet_ClassUSE Droplet_Class PRIVATEPRIVATE PUBLIC :: d_dtPUBLIC :: d_dt(), operator(*), operator(-)(), operator(*), operator(-)

TYPE CloudTYPE Cloud PRIVATE PRIVATE TYPE(Droplet), DIMENSION(:), ALLOCATABLE :: dropTYPE(Droplet), DIMENSION(:), ALLOCATABLE :: drop END TYPE Cloud END TYPE Cloud

INTERFACE d_dt INTERFACE d_dt MODULE PROCEDURE d_dt_CloudMODULE PROCEDURE d_dt_Cloud END INTERFACE END INTERFACE

CONTAINSCONTAINS

FUNCTION d_dt_Cloud(this) RESULT(dState_dt)FUNCTION d_dt_Cloud(this) RESULT(dState_dt) TYPE(Cloud), INTENT(IN) :: this TYPE(Cloud), INTENT(IN) :: this TYPE(Cloud), POINTER :: dState_dtTYPE(Cloud), POINTER :: dState_dt ......

DO i=1,nDO i=1,n dState_dt%drop(i) = d_dt(this%drop(i)) dState_dt%drop(i) = d_dt(this%drop(i)) END DOEND DO

InheritanInheritancece

PolymorphiPolymorphismsm

EncapsulatEncapsulationion


Fortran ImplementationFortran ImplementationMODULE Droplet_Class MODULE Droplet_Class

PRIVATE PRIVATE PUBLIC :: d_dt(), operator(*), operator(-)PUBLIC :: d_dt(), operator(*), operator(-)

TYPE Droplet TYPE Droplet PRIVATE PRIVATE REAL :: St REAL :: St REAL, DIMENSION(3) :: s REAL, DIMENSION(3) :: s REAL, DIMENSION(3) :: v REAL, DIMENSION(3) :: v END TYPE DropletEND TYPE Droplet

… …

CONTAINS CONTAINS FUNCTION d_dt_Droplet(this,u) RESULT(dState_dt) FUNCTION d_dt_Droplet(this,u) RESULT(dState_dt) TYPE(Droplet),POINTER :: dState_dtTYPE(Droplet),POINTER :: dState_dt TYPE(Droplet),INTENT(IN) :: thisTYPE(Droplet),INTENT(IN) :: this REAL ,INTENT(IN) ,DIMENSION(3) :: uREAL ,INTENT(IN) ,DIMENSION(3) :: u

… …

dState_dt%St = 0.0dState_dt%St = 0.0 dState_dt%s = this%vdState_dt%s = this%v dState_dt%v = (u-this%v)/this%StdState_dt%v = (u-this%v)/this%St


Fortran ImplementationFortran ImplementationMODULE Integrator_ClassMODULE Integrator_Class

USE Cloud_Class USE Cloud_Class USE Fluid_ClassUSE Fluid_Class

PRIVATE PRIVATE PUBLIC :: euler_IntegratorPUBLIC :: euler_Integrator

TYPE IntegrandTYPE Integrand PRIVATEPRIVATE TYPE(Cloud) , POINTER :: cloud_ptrTYPE(Cloud) , POINTER :: cloud_ptr TYPE(Fluid) , POINTER :: fluid_ptrTYPE(Fluid) , POINTER :: fluid_ptr END TYPE IntegrandEND TYPE Integrand…… CONTAINS CONTAINS SUBROUTINE euler_Integrator(this,dt)SUBROUTINE euler_Integrator(this,dt) TYPE(Integrand), INTENT(INOUT) :: thisTYPE(Integrand), INTENT(INOUT) :: this REAL(NBYTES) :: dtREAL(NBYTES) :: dt IF (ASSOCIATED(this%fluid_ptr) ) THENIF (ASSOCIATED(this%fluid_ptr) ) THEN

this%fluid_ptr = this%fluid_ptr + dt*d_dt(this%fluid_ptr)this%fluid_ptr = this%fluid_ptr + dt*d_dt(this%fluid_ptr)

ELSE IF (ASSOCIATED(this%cloud_ptr)) THENELSE IF (ASSOCIATED(this%cloud_ptr)) THEN……


Run-time Run-time polymorphismpolymorphismvia dynamic via dynamic dispatchingdispatching

Complete componentization of F90 codeComplete componentization of F90 code Study performance via fine-grained Study performance via fine-grained

parallelizationparallelization– Several recent developments presage a Several recent developments presage a

potentially renewed importance of OpenMP:potentially renewed importance of OpenMP: Multi-core chips (Intel)Multi-core chips (Intel) Hardware support for multithreading (Sun’s Hardware support for multithreading (Sun’s

Niagara)Niagara) Efficient emulation of SMP via NUMA clusters Efficient emulation of SMP via NUMA clusters

(SGI’s Altix)(SGI’s Altix)

Future WorkFuture Work


Date post:	31-Dec-2015
Category:	Documents
Upload:	nickolas-shelton
View:	213 times
Download:	0 times

12 March 2004 A High-Fidelity, High-Performance Turbulent Dispersion Framework Damian Rouson...

Documents