142 Los Alamos Science Number 29 2005

Direct Numerical Simulations of Turbulence

Data Generation and Statistical AnalysisSusan Kurien and Mark A. Taylor

In 1941, Andrei N. Kolmogorov predicted that, within allhighly turbulent flows, there is a universal energy-conservingcascade whereby the energy of the large-scale eddies is trans-ferred to finer and finer scales, down to the scales at which theenergy is finally dissipated to heat. It is difficult to measuresuch a cascade directly, but related benchmark predictions forthe statistical behavior of turbulent flows can now be calculatedand examined using advanced simulation and flow visualizationtools. Los Alamos scientists have been able to simulate flows ofReynolds numbers up to 105, the largest of which needed of theorder of terabytes of data storage and used the full power of the Advanced Simulation and Computing (ASC) Q machine for several weeks of computer time. Through clever analysis of single frames of the simulations, a great deal of informationcan be extracted to show that the original constraints for theKolmogorov theory can be relaxed so that, in fact, his statisticalpredictions hold locally in time. Furthermore, scientists are able to measure new statistical quantities that demonstrate the conditions under which departures from Kolmogorov theorybegin to occur. This type of statistical analysis of numericaldata is setting the agenda for future research.

Visualization of vorticity in a portion of a 2563 subdomain of the 20483

turbulence simulation performed on the ASC Q machine at Los Alamos.

The ASC Q machine.

The problem of fluid turbulencehas benefited from concertedefforts in theoretical, experi-

mental, and most recently, computa-tional research. However, whiletheoretical and experimental effortshave cooperated for some time toadvance the field, computational sci-ence is a relatively recent entry andprovides new data and problems thathave not been accessible by moreestablished techniques. For some prob-lems, the entire turbulent flow fieldcan now be calculated to high preci-sion with suitable numerical methods.Flow visualization and extensive three-dimensional (3-D) statistical analysis,for example, are techniques that can beused profitably. Computational capa-bilities and expertise at Los AlamosNational Laboratory have resulted incalculations that reveal new universalproperties of turbulence and newdirections in which to expand researchefforts, as we describe below.

Solving the Navier-Stokes equa-tions, which provide the best-knownmathematical description of turbulentflow, remains an immensely challeng-ing problem. However, turbulenceresearch is driven by a practical needfor real-world engineering applicationsand by the need to understand and pre-dict the universal fundamental fea-tures, if any, in all turbulentphenomena. Therefore, approaches tostudying turbulence other than compu-tational ones have evolved over sev-eral decades and have produced a deepunderstanding of the subject on a fun-damental as well as a phenomenologi-cal level. One such approach wasinitiated in the late 19th century byOsborne Reynolds, who proposed toignore the details of the turbulent flowat each instant and, instead, to regardthe flow as a superposition of meanand fluctuating parts. What naturallyfollowed this shift in approach was theaddition of statistics and probabilitytheory to the arsenal of tools used tounderstand turbulence. The turbulence

field is considered to be a random fieldin the probabilistic sense. The idea isto study the statistical moments of tur-bulent fields such as the multipointcorrelation functions of velocity, pres-sure, and so on with the aim to recoverthe full probability-distribution func-tion of the field and its evolution givena set of initial (boundary) conditions.Alternatively, there are attempts toobtain the probability distributionfunctions first and derive from themthe statistics of the turbulent system. In a broad sense, deriving these func-tions is the goal of statistical hydrody-namics research (refer to the article “ Field Theory and StatisticalHydrodynamics” on page 181 of thisvolume). This article will examinesome of the questions that statisticalanalysis of turbulence data can addressusing several data sets generated bysolving the Navier-Stokes equations ongrids with different spatial resolutions.

Universal Properties of Turbulence

First, we briefly address the prob-lem of universality of statistical prop-erties. We would like to knowwhether turbulence exists independ-ently of the type of flow (water flow-ing in a pipe or in a river, wind flow,and others), the fluid that is flowing(air or water), the boundary conditions(smooth, rough, artificial, or periodic),or the energy-input mechanisms (stir-ring, shaking, or shearing). Is there aregime of length scales that has quan-tifiable properties common to all tur-bulent flows? Two phenomenologicalideas have been useful in addressingthis question. The first was proposedby Lewis F. Richardson in the late19th century and is consistent withour intuition from observing turbu-lence—the energy input at large scalesis transferred into successively smallereddies of the turbulent flow in a so-called cascade process. The notion of

an eddy in turbu-lent

an “eddy” in turbulent flow is some-what nebulous, but for current pur-poses, it should be thought of as acoherent turbulence structure with anassociated length scale, location, andlifetime. The second idea is ahypothesis advanced by Andrei N.Kolmogorov (1941): For highly tur-bulent flows in which the Richardsoncascade has created many genera-tions of eddies, the turbulent lengthscales of size r that are much smallerthan the typical large scale L of theflow and much larger than the vis-cous dissipative scale η must haveuniversal statistical properties.Kolmogorov conjectured that, in thisregime of intermediate scales, thedynamics is minimally affected byforcing, boundaries, and large-scaleanisotropies, which are generallyflow-dependent, and unaffected bythe viscous dissipative effects thatoccur at the very small scales. Thedynamics in this so-called inertialrange are dominated by the nonlinearterm of the Navier-Stokes equations,and it seems reasonable that inertial-range dynamics should display uni-versal behavior statistically. In ourdiscussion of new statistical-analysisand diagnostic techniques, we will beconcerned primarily with the statis-tics of this universal inertial range ofscales in high-Reynolds-number tur-bulence (see the article “ TheTurbulence Problem” on page 124

Number 29 2005 Los Alamos Science 143

Direct Numerical Simulations of Turbulence

Volumerendering ofvorticity in the2563 subdomainshown on the opposite page.

for definitions of these terms).The typical statistical scale-

dependent quantities investigated areknown as structure functions, one typeof which is

(1)

where uL(x) = u(x)⋅r is the componentof the velocity along r (the subscriptL denotes longitudinal velocity) and⟨...⟩ denotes ensemble and domainaveraging over all x. This structurefunction is thus the nth-order momentof the velocity difference across scalesof size r and is a measure, order byorder, of the statistical properties ofeddies of size r. Kolmogorov deriveda fundamental physical law for theinertial range of scales r for highReynolds number, slowly decaying(essentially steady-state) turbulenceunder the assumption of isotropy andhomogeneity of the small scales:

(2)

where ε is the mean rate of energyflux balancing the mean rate of energydissipation in statistically steady turbu-lence in the limit of zero viscosity.This so-called “four-fifths law”(Kolmogorov 1941) is a statement ofenergy conservation in the inertialrange; that is, the energy flux throughscales of size r1 equals the energy fluxthrough scales of size r2 if both r1 andr2 are in the inertial range. The four-fifths law is now used as a nominalmeasure of the regime of inertial-rangescaling in experimental and numericaldata; that is, the range of scales overwhich the four-fifths law is close tobeing satisfied is taken to be the statis-tically “universal” scaling regime.

Kolmogorov also assumed that thecascade of energy occurs in a space-filling, self-similar way. Formally,there exists a unique scaling exponenth such that

(3)

To be consistent with the four-fifths law, the assumption of self-similarity implies that h = 1/3 andthat, in general, if structure functionsof arbitrary order are to scale with r,then

(4)

Most of the known empiricaldepartures from the Kolmogorov scal-ing prediction can be traced to threecauses: The Reynolds number is notlarge enough, the scaling is contami-nated by the anisotropies inevitable inmost flows, and the self-similarityassumption is not valid. The effectsrelating to small Reynolds numbersare something we have to live with, ina sense, because of the limitations oftechnology and computational power,but cumulative data analysis of exper-iments and simulations performedover several decades strongly suggestthat the scaling exponents do not dif-fer much for a Taylor microscaleReynolds number1 Rλ ranging fromapproximately 100 to approximately10,000.

It therefore seems that, at a mini-mum, we observe a convergence ofthe exponents over a wide range ofhigh Reynolds numbers. The assump-tion of statistical isotropy, that is,invariance under arbitrary rigid rota-tions, is key to the scaling-law predic-tions, but isotropy is a rather strongrestriction to make when most turbu-

lent flows are apparently highlyanisotropic. There are two ways toremove the inevitable effects ofanisotropy in order to test the funda-mental assumption of self-similarity.The first is to measure flows withextremely high Reynolds numbers,such as wind flow over the ground,that yield wide separation of scalesand resort to the Kolmogorov assump-tion that, for sufficiently small scales,the statistics will be locally isotropic.The second is to explicitly extract theisotropic component of the statistics,for example, by systematically averag-ing out the anisotropic contributions,as we discuss in detail below.Recently, the effect of anisotropy onscaling exponents has been studiedextensively, and there are now ways toquantify anisotropic effects (Kurienand Sreenivasan 2001), as well as toextract purely isotropic contributions(Taylor et al. 2003), which might thenbe more sensibly compared with theo-retical predictions. We will discuss anew method to implement the latterprocedure that has proved to be veryuseful in analyzing arbitrarilyanisotropic flows. The final knownreason for departure from theKolmogorov scaling prediction is thatthe turbulent cascade is not self-simi-lar. That is, instead of each generationof eddies being produced in a space-filling, self-similar way, the cascadeproceeds in an intermittent manner, inwhich some parts of the flow at agiven instant are extremely activewhile others are relatively quiescent.This is the now well-known intermit-tency feature of turbulence, and itresults in what is known as “anom-alous” scaling—that is, there is nounique scaling exponent h from which all scaling exponents can besimply derived.

In the remainder of this article, wedescribe our studies of the universalstatistical features of turbulence usingquantities such as the structure func-tions measured from simulations

144 Los Alamos Science Number 29 2005

Direct Numerical Simulations of Turbulence

1The Taylor microscale Reynolds number isRλ = u′λ/ν, where u′ is the velocity fluctua-tion and ν is the viscosity. Initially, G. I.Taylor thought that the scale λ—the radiusof curvature at the origin of the autocorrela-tion of the fluctuating velocity—was theviscous dissipation scale of turbulence. Infact, its magnitude is intermediate betweenthe large scale L and the true (Kolmogorov)dissipation scale η. Rλ is often used insteadof the large-scale Reynolds number, Re, tocharacterize flows that have widely varyinglarge-scale properties and, hence, widelyvarying Reynolds numbers but whose small-scale fluctuations might be comparable. Athigh Reynolds numbers, Rλ ∝ Re1/2.

(resolved down to the dissipationscale) of the fundamental equations ofmotion, the Navier-Stokes equations.First, we discuss the simulationsthemselves and then demonstrate theuse of diagnostics to extract statisti-cally isotropic features of the flow.Our results suggest a refinement ofthe Kolmogorov picture of isotropicturbulence.

Direct Numerical Simulations

Direct numerical simulation (DNS)refers to solving the Navier-Stokesequations numerically by resolving allscales down to the scale of viscousdissipation. DNS represents a brute-force approach to modeling turbu-lence: No modeling is requiredbeyond the Navier-Stokes equations,simple well-understood numericalmethods are used, but massive com-puting resources are needed. Whencarefully produced, DNS data is anexcellent substitute for exact, analyticsolutions of the Navier-Stokes equa-tions. The only drawback is that toobtain solutions for moderately highReynolds numbers requires weeks ofcomputing time on today’s largestsupercomputers. To achieve theReynolds numbers of a typical atmos-pheric boundary layer flow, Rλ= 10,000, will require a 108-foldincrease in computing power overtoday’s largest computers. Fortunately,large-scale features such as the meanflow and other statistical properties ofturbulence depend only weakly on theReynolds number. Thus, DNS offlows with more moderate Reynoldsnumbers has been valuable for study-ing many aspects of turbulence,including universal statistical features.For additional information, see, forexample, the review by Moin andMahesh (1998).

To obtain as high a Reynolds num-ber as possible, DNS calculations areusually performed on the simplest

flows: the incompressible Navier-Stokes equations, without multiplematerials or other physics that mustbe modeled. The calculations are fur-ther limited to simple domains andequally spaced grids, which allow forvery efficient numerical algorithms.The highest possible Reynolds num-bers can be achieved for the classicproblem of homogeneous turbulencein a square box with periodic bound-ary conditions, the problem we havefocused on.

For fully resolved calculations,spectral methods are preferred fortheir high accuracy. Although high-order finite-difference codes can yieldsimilar accuracy, spectral methodsstill have an advantage because theypermit fast, direct solution ofPoisson’s equation. Solving Poisson’sequation is required to determine thepressure gradient that appears in theNavier-Stokes equations. Spectralmethods became practical for compu-tational fluid dynamics after thedevelopment of the spectral-transformmethod (Eliasen et al. 1970, Orszag1970). Additional issues important forthe Navier-Stokes equations, such astime-stepping schemes and control ofaliasing errors, were effectivelytreated in Rogallo (1981). The meth-ods used today are quite similar tothose used in that work.

The spectral part of a DNS coderefers to the method used for the spa-tial discretization of the equations. Inparticular, to compute a spatial deriva-tive of a term in the equations, onefirst expands that term in a truncatedFourier expansion using the fastFourier transform (FFT) and thencomputes the derivatives exactly fromthe Fourier expansion. After the equa-tions are discretized in space, we areleft with a system of ordinary differ-ential equations, which are integratedin time with a third- or fourth-orderRunge-Kutta or similar scheme. Thisprocedure has one complication aris-ing from the nonlinear advection term.

The nonlinearity can transfer energyinto frequencies higher than can beresolved by the numerical grid. Theenergy in these unresolved frequen-cies will then artificially contaminatethe energy and phases of the resolvedfrequencies in a procedure known asaliasing. This aliasing error is typi-cally controlled by properly designedspectral filters.

The computational expense ofDNS comes from the strict restrictionson the grid spacing, ∆x, and the timestep, ∆t, that are required to fullyresolve all scales in the Navier-Stokesequations. If one is primarily inter-ested in the statistical properties of theinertial range, it is sufficient to run thenumerical simulation with ∆x ≤ 3η,where η is the Kolmogorov lengthscale.2

Since η ∼ Re–3/4 (or η ∼ Rλ–3/2),

this grid-spacing restriction also deter-mines the highest-Reynolds-numberflow that can be accurately computedfor a given ∆x. The restrictions on ∆tcan be estimated from the considera-tion of physical time scales in theproblem, but in practice, a morerestrictive constraint comes from theCFL (Courant-Friedrichs-Levy) condi-tion, which shows that, for the time-stepping schemes used, the time stepmust be kept proportional to the gridspacing. Combined, these considera-tions show that the computational costof DNS is proportional to Rλ

6 (Pope2000).

In DNS calculations, it is importantto ensure that the energy dissipation isdue entirely to the viscous terms inthe Navier-Stokes equations, rather

Number 29 2005 Los Alamos Science 145

Direct Numerical Simulations of Turbulence

2 The Kolmogorov length scale η dependsonly on the rate of energy flux ε and the(chosen) fluid viscosity ν. In the forcedsimulations, η is determined entirely bythe forcing (rate of input of energy),which balances the flux rate in the statisti-cal steady state and the chosen viscositycoefficient. In the decaying simulation, ηis fully determined at initial time by theinitial condition but thereafter evolveswith the dynamics, thus resulting inincreasing resolution as the flow decays.

than to the numerical method used.Often, numerical methods aredesigned to introduce various types ofartificial dissipation, which can havebeneficial properties but are notappropriate for DNS. For the spectralmethod outlined here, we estimate thenumerical viscosity by computing thekinetic energy E at every time stepand comparing the numerical evolu-tion of E,

where E = 0.5 ⟨u⋅u⟩, with the evolu-tion given by the Navier-Stokes equa-tions. In the unforced case, the latterterm is

where u is the flow field. In ourlargest simulation, the two quantitiesagree to more than four digits, demon-strating that over 99.99 percent of thedissipation is due to the Navier-Stokesviscosity.

Finally, if DNS in a periodic boxis used to study universal features ofturbulence, the largest scales arestrongly influenced by the squarecomputational domain. For example,consider a field with all its energy inspherical wave numbers of at most2. There are only a handful of suchFourier modes, and they are stronglyaligned with the coordinate direc-tions of the box. Any such fieldcould not be isotropic. Many of thedirectional moments of the fieldwould greatly differ between coordi-nate and noncoordinate directions.There are several ways to avoid thiseffect in order to obtain moreisotropic simulations. The mostdirect method is to simply keepenergy out of the large scales. Thisis the approach usually taken withdecaying turbulence simulations. Forforced simulations, it is possible toachieve flows with much higherReynolds numbers by injecting

energy into only the low wave num-bers, but to obtain isotropic solutionsrequires careful attention. Oneapproach is to use stochastic forcingdesigned so that the flow will beisotropic for a long enough timeaverage, even though the field at anygiven time will have largeanisotropies at the large scales. Thisapproach introduces a lot of fluctua-tions in the solutions, so long timeaverages must be taken to obtainconverged statistics. The most effi-cient approach is to use smooth,deterministic low-wave-number forc-ing. Converged statistics can then beobtained with shorter time averages,but some anisotropy will persistthroughout the flow. For many quan-tities of interest, however, thisanisotropy can be removed with theangle-averaging techniquesdescribed below.

In our work, we have examinedDNS simulations for decaying turbu-lence, stochastically forced turbu-lence, and deterministically forcedturbulence (refer to Table I). For thedecaying turbulence simulations, aproperly chosen initial condition isallowed to decay through the effectsof viscosity. For the forced prob-lems, the simulations are run untilthe forcing and dissipation reach sta-tistical equilibrium, and then theyare run for several additional eddy

turnover times to collect data fromthe equilibrium regime.

The decaying problem has theadvantage that more realistic flowscan be simulated, and it is possible, inprinciple, to compare the simulationresults with those from experiments,such as those carried out at therecently upgraded Corrsin WindTunnel (Kang et al. 2003). But thedecaying problem has the drawbackthat the results strongly depend on theinitial condition, and one is faced withthe challenge of generating a realisticturbulence state to use for the initialcondition. To address this problem, indata set 5, we have followed the pro-cedure described by Kang et al.(2003). We generate an initial flowfield with random, uncorrelatedphases but a prescribed energy spec-trum. The flow is then run for a shorttime, until the phases become corre-lated enough to give a reasonablemean-derivative skewness. Theenergy spectrum is then reinitializedback to the original spectrum whileretaining the correlated phases. Ourlow-wave-number forcing schemesare described in detail in Taylor et al.(2003). The deterministic forcing isbased on the work by Sullivan et al.(1994), Sreenivasan et al. (1996), andOverholt and Pope (1998). Data sets 1and 4 were obtained with this forcing.Data set 3 was obtained with a similar

146 Los Alamos Science Number 29 2005

Direct Numerical Simulations of Turbulence

scheme, but modified to inject helicityinto the flow. The stochastic forcingused for data set 2 was based on theforcing given in Gotoh et al. (2002).We used both types of forcing todemonstrate the equivalence of theresults when angle averaging isapplied to the data.

Parallel Computing Issues

DNS calculations at resolutions ofup to 5123 can now be obtained onmoderately large clusters. But thelarger DNS calculations currentlyrequire Advanced Simulation andComputing (ASC)-class supercomput-ers. Our largest simulation, with a res-olution of 20483, requires a 256-foldincrease in computing power over thatrequired for a resolution of 5123. With8 billion grid points, our 20483 simu-lation is one of the largest ever com-pleted. It required several weeks using2048 processors of ASC-Q and wasmade as part of the Laboratory’sScience Runs to showcase ASC-Q’sperformance.

To implement FFT-based DNScodes on distributed memory parallelcomputers, the community reliesalmost exclusively on the data-trans-pose method. Each processor mustperform thousands of FFTs per timestep, but the data required for thoseFFTs will be distributed among manyother processors. It is quite difficultto write an efficient, distributed-dataFFT, and thus the data-transposemethod continuously adjusts the dis-tribution of data among the proces-sors so that each processor can use aconventional serial FFT. The name“transpose” comes from the fact thatif the data distribution is representedon a 3-D mesh of processors, theoperations required by the data-trans-pose algorithm look like matrixtransposes. For a resolution of 20483,over a terabyte of data must bemoved through the network for each

time step, and thus the method relieson a tightly coupled parallel com-puter with very high bandwidth. OnASC-Q, for problems of size N3, weobtain excellent scaling for up to N/2processors. Using N processors stillrepresents a significant speedup, butthe scalability starts to decrease, sothere is little benefit to using morethan N processors.

Another important problem con-cerns data input/output (I/O). For aresolution of 20483, each flow snap-shot (which can also be used as arestart file) is 192 gigabytes. SerialI/O (having a single processor collectthe data from all other processors andwrite it into a single file) can obtaindata rates only in the tens ofmegabytes per second and thusrequires hours to write a single snap-shot or read in a snapshot whenrestarting. To avoid this unacceptablebottleneck, we utilized the UnifiedData Model (UDM) I/O library of theHigh-Performance ComputingEnvironment Group at Los Alamos.UDM, in conjunction with ASC-Q’sparallel-file system, allows allprocessors to participate in the I/Ofor a single file. With UDM, we wereable to obtain data-transfer rates ofover 500 megabytes per second,which means snapshots can be writ-ten or read in under 7 minutes.

The Angle-AveragingTechnique

In general, the two-point structurefunction S(r) defined in Equation (1)is a function of the vector r, that is, afunction of the size of the separationscale r = |r|, as well as of the orienta-tion of r. The Kolmogorov 1941 the-ory, however, assumes that, forsufficiently small scales, the flowdepends only on the magnitude of rand is independent of the orientationof r. Most reasonably controlled flowexperiments (for example, those

occurring in wind tunnels or pipes),as well as uncontrolled experiments(for example, those involving meas-urements of velocity in the atmos-pheric boundary layer), inevitablyhave some degree of anisotropyeither from boundary configurationsor from forcing mechanisms.Therefore, reasonable comparisonswith theoretical predictions requireunderstanding the degree of contami-nation caused by arbitrary anisotropyas well as formulating methods toeliminate these effects from the data.From experiments at very highReynolds numbers (Taylor Reynoldsnumber of ~10,000 or higher), inwhich there is wide separationbetween the large scales and the dis-sipative scales, we know that, fortwo-point statistics of the structurefunctions given in Equation (1), thecontamination due to anisotropydecays rapidly with scale size andthat local isotropy is recovered in theleading order. In numerical simula-tions, the Reynolds numbers, as wellas the range of scales computed, aremuch smaller, and anisotropic effectstypically do not have enough rangeof scales to decay sufficiently. As aresult, they have a significant contri-bution in the inertial range. However,the availability of the full spatial andtemporal information of the flowfield offers other unique possibilitiesfor investigating purely isotropiceffects. One general procedurerecently developed at Los Alamos isthe angle averaging of the structurefunctions, which averages out theanisotropic contributions of an arbi-trary (anisotropic) flow.

The primary motivation for ourangle-averaging procedure is therecent derivation of a new version ofthe Kolmogorov four-fifths law(Duchon and Robert 2000, Eyink2003). In this version, the four-fifthslaw states that for any domain B ofsize R in the limit that the viscosityν → 0 (infinite or sufficiently high

Number 29 2005 Los Alamos Science 147

Direct Numerical Simulations of Turbulence

Reynolds number), for scales of sizer << R, and at any instant in time,

(5)

where εB is the energy dissipation rateaveraged over B. That is, the four-fifths law holds locally, instanta-neously, and without any assumptionof homogeneity or isotropy. The inte-gration over the solid angle Ω, indi-cates averaging over all possibleorientations of r for a given |r|, whichprojects out the isotropic part of thecorrelation. The statement of energyconservation in the inertial range isnow quite different—there is an under-lying isotropic component common toall flows that formally obeys the samelaw that Kolmogorov derived usingmore restrictive assumptions.

To test this prediction with numeri-cal simulations, we devised a way totake the solid-angle average of thedata computed on a grid. The obvious,but computationally expensive anderror-prone solution, would be tointerpolate the velocity vector fieldover a sphere of desired radius r andintegrate. Instead, we chose to firstuse the separation vectors allowed bythe grid to compute structure func-tions for a fixed (θ,ϕ) as a function ofr, as follows:

Then, we computed a set of thesestructure functions for various (θ,ϕ)allowed by our grid so that we have aset of functions S(r,θ1,ϕ1),S(r,θ2,ϕ2),… S(r,θn,ϕn) for pairs ofangles (θi,ϕi) that span the full spheri-cal solid angle rather uniformly. EachS is now a smooth function of r in aparticular direction and can be inter-

polated to obtain S(r) for any r. Then,to yield the angle-averaged value for aparticular r, we compute

(6)

where the weight wi is the solid anglesubtended by the Voronoi cell contain-ing the point r.

As n → ∞, the average becomesarbitrarily close to the true sphericalintegral of Equation (5), and so theisotropic component of the statistics isrecovered. The method is not specificto the four-fifths law and can in prin-ciple be used to examine the underly-ing isotropic component of anytwo-point correlation function, as wedemonstrate below.

The Four-Fifths Law

Figure 1 shows such a calculationperformed on a single frame of ananisotropically forced flow at a reso-lution of 1024 grid points to a sidewith periodic boundary conditions

(data set 4), which was run longenough to achieve a statisticallysteady state. Each colored line is thecompensated, domain-averaged, lon-gitudinal third-order structure func-tion, S3(r)/εr, computed in aparticular direction in the periodicbox for the increments r allowed bythe grid in that direction. The lengthscale r has been nondimensionalizedby the dissipation length scale η. Thecompensated statistics were com-puted in 73 different directions thatwere fairly evenly distributed overthe sphere. As is clearly seen, thecalculation in a given direction yieldsa smooth curve, which we interpo-lated using a cubic spline fit toobtain S3(r)/εr for arbitrary length rin a given direction. The differentdirections also clearly display a largedegree of variability with respect toeach other, which appears to dimin-ish as the scales get very small but issignificant in a midrange of scaleswherein the inertial range might beexpected to lie. The thick black lineis the average over all 73 directionsof S3(r)/εr as a function of r calcu-

148 Los Alamos Science Number 29 2005

Direct Numerical Simulations of Turbulence

Figure 1. The Four-Fifths Law for a Single Frame of Forced FlowThe four-fifths law was computed for a single frame of data set 4 for deterministicforced flow, whose resolution is 10243. Each colored line is the compensated third-order structure function computed in one of 73 different directions of the flow. Theblack line is the angle-averaged function, which displays a range of scales between30 and 200 that fall within 5% of the theoretical value of 0.8.

lated according to Equation (6).Remarkably, this angle-averaged

function displays a reasonable rangeover which the curve is rather flat(indicating linear scaling in r) and iswithin 5 percent of 0.8, which is thetheoretical predicted value. Thisresult says that, at every instant in ananisotropic flow, there is an underly-ing isotropic component that can beprojected out when an approximatedspherical average is used and that,furthermore, obeys to a very gooddegree the fundamental universalfour-fifths law for isotropic flow.

In Figure 2, we show the same cal-culation for data sets 1 and 2, whichwere calculated at lower Reynoldsnumbers but are forced in the lowwave numbers as described above.The solid (black) and dotted (red)lines are the angle-averaged and thentime-averaged compensated third-order structure functions for data sets1 and 2, respectively. While the scal-ing range for this resolution is less

than that in Figure 1, the noteworthyfeature is that the curves are indistin-guishable, which is a strong indicationof universality because the underlyingisotropic contributions of these twovery different anisotropic flows areidentical (Taylor et al. 2003).

The Two-Fifteenths Law

To demonstrate the distinctionbetween the Kolmogorov localisotropy assumption and what we seein Figure 1, we discuss the measure-ment, using the same angle-averagingtechnique, of a very different statisti-cal quantity that obeys the so-calledtwo-fifteenths law:

where h is the mean helicity dissipa-tion rate and uT denotes the compo-

nent of u(r) transverse to r. The quan-tity on the left side of this equation isa third-order statistic, as is S3(r) forthe four-fifths law, but this new quan-tity probes the presence of a constanttotal helicity flux h in the inertialrange (Kurien 2003). Like energy,helicity (u ⋅ ∇ × u) is conserved inturbulence, and our analysis hasrevealed that in the inertial range,helicity has other conserved propertiesin common with those of energy, suchas constant flux.

Figure 3 shows this parity-breakingthird-order statistic normalized by hr2

in a forced flow in a periodic box of512 grid points to a side with fixedsign of helicity input into the twolowest modes at each time step (dataset 3). The picture in Figure 3 indi-cates that helicity flux (that is, theappropriate third-order correlationfunction) is highly anisotropic all theway into the small scales, as shown bythe vast spread among the differentdirections. Nevertheless, there is still

Number 29 2005 Los Alamos Science 149

Direct Numerical Simulations of Turbulence

Figure 2. Angle- and Time-Averaged CompensatedThird-Order Structure Function for Two DifferentForced FlowsThe angle- and time-averaged compensated third-orderstructure function was computed for data sets 1 (solid line)and 2 (dotted line), each of which has a resolution of 5123.These two differently forced flows essentially coincide witheach other in this statistical measure, thus supporting thenotion of underlying universality of turbulent flows.

Figure 3. The Two-Fifteenths Law from a Single Frameof Data Set 3The two-fifteenths law was computed for a single frame of dataset 3, whose resolution is 5123. Each colored line is the com-pensated third-order statistic in one of 73 different directions inthe flow. The black curve is the angle-averaged function, whichshows a range between 30 and 200 wherein its value is within4% of the theoretically predicted value of 2/15.

an underlying isotropic component(thick black line) that emerges fromthe angle-averaging procedure andseems to agree with the universallypredicted two-fifteenths law towithin 5 percent over a reasonablerange of scales. This analysis(Kurien et al. 2004) reveals that theflux of helicity is more anisotropicand intermittent (in the sense oflarge departures from the mean) thanthe energy flux measured analo-gously by the four-fifths law(Figures 1 and 2).

In summary, angle averaging andstatistical analysis have revealed thatthe isotropic component in turbulentflows is universal, agrees rather wellwith the Kolmogorov theory, andmoreover, is consistent with the

local version of Duchon and Robert(2000) and Eyink (2003). The proce-dure allows us to separate the con-tamination due to anisotropy fromother effects, such as small Reynoldsnumber and intermittency, that canmuddy the measurement of cleanscaling laws. The angle-averagingmethod also gives us a way to moreefficiently use data and gain statisti-cally significant results from singlesnapshots of the flow, whereas in thepast, long time averages were taken,which led to data size and storageissues. Especially when we begin tostart looking at the storage andanalysis of data set 5, which needsof the order of 250 gigabytes of diskspace, a scheme such as the angle-averaging procedure, which

increases the amount of informationwe can glean from a single frame ofturbulence data, is a definite asset.

Utility of Large-ScaleSimulations

Our largest simulation (data set 5) isfor a very highly resolved (20483),decaying flow at a moderate Reynoldsnumber (270). The simulation’s initialcondition was taken from the centerlinedata gathered from a wind tunnel exper-iment performed at Johns HopkinsUniversity (Tao et al. 2000). The simu-lation, performed on 2048 processors ofASC-Q, did not achieve the Rλ ∼ 700 ofthe experiment. Therefore, direct com-parison with the experimental resultscannot be made until we can computedecaying flow at a higher Reynoldsnumber or the experimental facility canrerun the experiment at a Reynoldsnumber matching that of the existingsimulation. However, a full numericalsimulation provides access to the fullspatial and temporal velocity field,while the experiments normally meas-ure a sparse subset of the flow field.

Figure 4 shows the surfaces of con-stant vorticity magnitude for a 2563

subdomain of the 20483 simulation.Vorticity visualizations are typicallyused to show the locations of the flowstructures. In this case, the vorticityvisualization shows that the generationof successively smaller energetic struc-tures occurs by the stretching of regionsof vorticity by the nonlinearity. Thesmall structures in Figure 4 persistdown to the grid size of the simulation.

Data sets 1 and 2 had Reynoldsnumbers similar to the number for dataset 5 but only a quarter of the numberof grid points to a side. That is, the lin-ear size of the smallest scales resolvedin the 5123 simulations of data sets 1and 2 was 64 times larger than thesmallest scales in the 20483 simulationof data set 5. Because the 5123 simula-tions cannot resolve almost two orders

150 Los Alamos Science Number 29 2005

Direct Numerical Simulations of Turbulence

Figure 4. Surfaces of Constant Vorticity for Decaying TurbulenceThis visualization is of the surfaces of constant vorticity magnitude in one of the2563 subdomains of the entire 20483 simulation (data set 5). There are 512 such sub-domains in this simulation.

of magnitude in scale that are accessibleto the 20483 simulation, the coarsersimulations obscure the turbulent finestructure seen at higher resolutions.Although they are quite suitable forobserving the many inertial-range fea-tures described above, the coarser com-putations obscure the significantenergetic events that occur at higherresolution. Clearly if we are to gain adeeper understanding of the spatial andtemporal universal properties of turbu-lence through such numerical calcula-tions, we must continue to pursue waysto compute larger resolved Navier-Stokes simulations and to develop effi-cient methods for analyzing theenormous quantities of data involved. n

Further Reading

Duchon, J., and R. Robert. 2000. InertialEnergy Dissipation for Weak Solutions ofIncompressible Euler and Navier-StokesEquations. Nonlinearity 13: 249.

Eliasen, E., B. Machenhauer, and E.Rasmussen. 1970. On a Numerical Methodfor Integration of the HydrodynamicalEquations with a Spectral Representation ofthe Horizontal Fields. In Report No. 2.Institute for Theoretical Meteorology,University of Copenhagen

Eyink, G. L. 2003. Local 4/5-Law and EnergyDissipation Anomaly in Turbulence.Nonlinearity 16: 137.

Gotoh, T., D. Fukayama, and T. Nakano. 2002.Velocity Field Statistics in HomogenousSteady Turbulence Obtained Using a High-Resolution Direct Numerical Simulation.Phys. Fluids 14 (3): 1065.

Kang, H. S., S. Chester, and C. Meneveau.2003. Decaying Turbulence in an Active-Grid-Generated Flow and Comparisons withLarge-Eddy Simulation. J. Fluid Mech. 480:129.

Kolmogorov, A. N. 1941. The Local Structureof Turbulence in Incompressible ViscousFluid for Very Large Reynolds Numbers.Dok. Akad. Nauk. SSSR 30: 301.

Kurien, S. 2003. The Reflection-AntisymmetricCounterpart of the Kármán-HowarthDynamical Equation. Physica D 175 (3–4):167.

Kurien, S., and K. R. Sreenivasan. 2001.Measures of Anisotropy and the UniversalProperties of Turbulence. In New Trends inTurbulence: Turbulence Nouveaux Aspects:École de Physique DES Houches—Ujf andInpg—Grenoble, a NATO Advanced StudyInstitute, Les Houches, Session LXXIV, 31July–September 1, 2000. p. 53. Edited byM. Lesieur, and F. David. New York:Springer-Verlag.

Kurien, S., M. A. Taylor, and T. Matsumoto.2004. Isotropic Third-Order Statistics inTurbulence with Helicity: the 2/15-Law. J. Fluid Mech. 515: 87.

Moin, P., and K. Mahesh. 1998. DirectNumerical Simulation: A Tool forTurbulence Research. 1998. Annu. Rev.Fluid Mech. 30: 539.

Orszag, S. A. 1970. Transform Method for theCalculation of Vector-Coupled Sums:Application to the Spectral Form of theVorticity Equation. J. Atmos. Sci. 27 (6):890.

Overholt, M. R., and S. B. Pope. 1998. ADeterministic Forcing Scheme for DirectNumerical Simulations of Turbulence.Comp. Fluids 27 (1): 11.

Pope, S. B. 2000. Turbulent Flows. Cambridge,United Kingdom: Cambridge UniversityPress.

Rogallo, R. S. 1981. Numerical Experiments inHomogeneous Turbulence. NASA TechnicalReport TM81315.

Sreenivasan, K. R., S. I. Vainshtein, R.Bhiladvala, I. San Gil, S. Chen, and N. Cao.1996. Asymmetry of Velocity Increments inFully Developed Turbulence and theScaling of Low-Order Moments. Phys. Rev.Lett. 77 (8): 1488.

Sullivan, N. P., S. Mahalingam, and R. M. Kerr.1994. Deterministic Forcing ofHomogeneous, Isotropic Turbulence. Phys.Fluids 6 (4): 1612.

Tao, B., J. Katz, and C. Meneveau. 2000.Geometry and Scale Relationships in High Reynolds Number Turbulence Determinedfrom Three-Dimensional HolographicVelocimetry. Phys. Fluids 12 (5): 941.

Taylor, M. A., S. Kurien, and G. L. Eyink.2003. Recovering Isotropic Statistics inTurbulence Simulations: The Kolmogorov4/5th Law. Phys. Rev. E 68 (2): 26310.

Number 29 2005 Los Alamos Science 151

Direct Numerical Simulations of Turbulence

For further information, contact Susan Kurien (505) 665-0148(skurien@lanl.gov) or Mark Taylor (505)284-1874 (mataylo@sandia.gov).