Verification of PDE Discretization Library and FlowApplication Codes

Roy H. Stogner

The University of Texas at Austin

September 30, 2009

Roy H. Stogner PDE Verification September 30, 2009 1 / 13

The Problem

3D Hypersonic Flow Approximations

Math, Software Components• Discretized Formulation

• Spatial Discretization

• Time Discretization

• System Assembly

• Nonlinear Solver

• Linear Solver

• I/O

• Postprocessing

Approximate LoC

DPLR 46,000Coupling 6,000Ablation 4,000Radiation 1,000FIN-S 6,000libMesh 54,000PETSc 170,000Other 56,000

Roy H. Stogner PDE Verification September 30, 2009 2 / 13

The Problem

3D Hypersonic Flow Approximations

Components To Verify• Discretized Formulation

• Spatial Discretization

• Time Discretization

• System Assembly

• Nonlinear Solver

• Linear Solver

• I/O

• Postprocessing

Potentially Buggy LoC

DPLR 46,000Coupling 6,000Ablation 4,000Radiation 1,000FIN-S 6,000libMesh 54,000PETSc 170,000Other 56,000

Roy H. Stogner PDE Verification September 30, 2009 2 / 13

Code Verification Library Verification

Regression Tests

Revise Software, Rerun Tests• Example applications, unit tests, benchmark tests

• Catches changes’ unintended consequences• Continuous Build System automation

I Tests not just run “by hand” by developersI libMesh, application tests in Trac/Bitten at INLI libMesh, FIN-S basic tests now in BuildBot at UTI Ablation, radiation, QUESO tests in BuildBot

Unit Tests• Written to test one object at a time

I Reusable modules interact with all other code through a limited APII That API can be tested directly outside of application codeI Test one method at a time, isolate problems locallyI 108 Unit Tests in libMesh

Roy H. Stogner PDE Verification September 30, 2009 3 / 13

Code Verification Library Verification

Regression Tests

Revise Software, Rerun Tests• Example applications, unit tests, benchmark tests

• Catches changes’ unintended consequences• Continuous Build System automation

I Tests not just run “by hand” by developersI libMesh, application tests in Trac/Bitten at INLI libMesh, FIN-S basic tests now in BuildBot at UTI Ablation, radiation, QUESO tests in BuildBot

Unit Tests• Written to test one object at a time

I Reusable modules interact with all other code through a limited APII That API can be tested directly outside of application codeI Test one method at a time, isolate problems locallyI 108 Unit Tests in libMesh

Roy H. Stogner PDE Verification September 30, 2009 3 / 13

Code Verification Library Verification

High-level Assertions

Manual assert(), PETSc debug mode• Active only in “debug” runs

• Verifies function preconditions, postconditions

• Approx. 4000 assertions in libMesh, FIN-S

• Can provide per-processor stack traces, etc.

Low-level Assertions• Defining GLIBCXX DEBUG

I Runtime bounds-checking of standard vector, iterator use

• ifort “-check bounds”

• Valgrind memory testing

• Out Of Bounds errors can lead to corrupt data, not just segfaults!

Roy H. Stogner PDE Verification September 30, 2009 4 / 13

Code Verification Library Verification

Assertions Have Uncovered MANY Library Bugs

• Uninitialized data

• Unpartitioned elements

• “Tearing” in neighbor map reconstruction

• Parallel vector operation miscommunication

• Parallel mesh adaptivity synchronization failures

• Out-of-order API calls

• Nelem < Nproc bugs

• Bad I/O node numbering

• Unsupported I/O format features

• Failure to “deep copy” a Mesh

And COUNTLESS Application Bugs

• API mistakesI Array misallocationI Misordered function argsI Sharing non-shareable objectsI Querying data before calculating itI Off-by-one, transposed indexing errors

I Finite elements on incompatible geometric

elements

• Runtime problemsI Invalid input filesI Unsupported p-refinement levelsI Attempting incompatible mesh

modification

Roy H. Stogner PDE Verification September 30, 2009 5 / 13

Code Verification Library Verification

Assertions Have Uncovered MANY Library Bugs

• Uninitialized data

• Unpartitioned elements

• “Tearing” in neighbor map reconstruction

• Parallel vector operation miscommunication

• Parallel mesh adaptivity synchronization failures

• Out-of-order API calls

• Nelem < Nproc bugs

• Bad I/O node numbering

• Unsupported I/O format features

• Failure to “deep copy” a Mesh

And COUNTLESS Application Bugs

• API mistakesI Array misallocationI Misordered function argsI Sharing non-shareable objectsI Querying data before calculating itI Off-by-one, transposed indexing errors

I Finite elements on incompatible geometric

elements

• Runtime problemsI Invalid input filesI Unsupported p-refinement levelsI Attempting incompatible mesh

modification

Roy H. Stogner PDE Verification September 30, 2009 5 / 13

Code Verification Library Verification

Parametric Testing

One Test Code, Many Tests• Keep test codes generic

• Execute with many different parameter choices• Compile time examples:

I Algebraic solver interfaceI Real/complex arithmeticI Mesh data structure

• Run time examples:I Mesh geometry, element typeI Discretization type, orderI Partition type, processor countI Error indicator typeI Adaptive refinement strategyI I/O formatI Coupled physics model version

Roy H. Stogner PDE Verification September 30, 2009 6 / 13

Code Verification Application Verification

Verification Benchmark Problems

Choosing Test ProblemsCapitalize on anything you know a priori:

• Known/manufactured solutionsI Exact solution to discretized problemI Limit solution of continuous problemI Solution from alternative code,

reduced modelI Known quantities of interest,

symmetries

• Known asymptotic convergence rates

• Known residuals

101

102

103

104

105

10−6

10−5

10−4

10−3

10−2

10−1

100

101

Number of Degrees of Freedom

Err

or F

ract

ion

Error Convergence, Manufactured Benchmark

Powell−Sabin 6−split L2 error

Powell−Sabin 6−split H1 errorPowell−Sabin 12−split L

2 error

Powell−Sabin 12−split H1 errorClough−Tocher 3−split L

2 error

Clough−Tocher 3−split H1 error

Roy H. Stogner PDE Verification September 30, 2009 7 / 13

Code Verification Application Verification

Jacobian Verification

J ≡ ∂R∂u

Residual, Jacobian Construction• Inherently physics-dependent

• But can be validated against each other

Finite Differencing

Jij ≈Ri(u + εej)−Ri(u− εej)

2ε

Greedy or element-wisealgorithms handle sparsity

Complex-Step Perturbations

Jij ≈Im[Ri(u + εej

√−1)]

ε

Avoids floating point subtractivecancellation error

Automatic Differentiation• Variable constructors seed derivatives

• Variable operations evaluate derivatives

Roy H. Stogner PDE Verification September 30, 2009 8 / 13

Code Verification Application Verification

Manufactured Solution Example

Convection-Diffusion Problem with AdjointsResidual equation:

R(u) = ∇ · α∇u + β~ex · ∇u + f = 0

Manufactured solution:

u ≡ 4(1− e−αx − (1− e−α)x)y(1− y)

• Homogeneous Dirichlet boundary

• α controls flux strength, layer

• Choose any convection strength β, solve for f• β = 0 gives simple series adjoint solutions

Roy H. Stogner PDE Verification September 30, 2009 9 / 13

Code Verification Application Verification

Verification Examples

Goal-Oriented Refinement• Adjoint-based error indicator weighting

I New libMesh featureI Test against manufactured solution

problems first

• Convergence “plateaus” were found inmultiple refinement strategies

• UniformRefinementEstimatorrequired new code to solve for adjointsolution errors

• PatchRecoveryErrorEstimatorrequired new seminorm integration (H1

vs. W 1,inf ) to give compatible errorsubestimates

Roy H. Stogner PDE Verification September 30, 2009 10 / 13

Code Verification Application Verification

Verification Examples

Parameter Sensitivity• Adjoint-based physics-independent

discrete sensitivity calculationsI New libMesh featureI Tested with manufactured solutions

• Convergence to analytic sensitivityplateaus at 2% relative error in everyrefinement strategy

• Finite differenced partial derivatives notresponsible

• Manufactured solution allowedsensitivity subcomponent comparisonto analytic solutions

• Sign errors in libMesh parametersensitivity method

5 6 7 8 9 10 11 12−4.2

−4

−3.8

−3.6

−3.4

−3.2

−3

−2.8

−2.6

Log(Dofs), Degrees of Freedom

Log(

Rel

ativ

e E

rror

)

Uniform (10−4)

ARPP (10−0)

ARPP (10−2)

ARPP (10−4)

Roy H. Stogner PDE Verification September 30, 2009 11 / 13

Code Verification Application Verification

Verification Examples

3 3.5 4 4.5 5 5.5−4

−3.5

−3

−2.5

−2

−1.5

Log(Dofs), Degrees of Freedom

Log(

Rel

ativ

e E

rror

)

Uniform (10−6)

ARPP (10−6)

Parameter Sensitivity Verification• “Off by 100%” error remaining in

one small solution term

• Switch to u′′ = f , 1D quadraticsolutions, manufactured residualtest

• Identified bug in repeatedadjoint solve rhs assembly

• Returning to manufactured solutionbenchmark: now converges to truesolution

Verification ContinuesRoy H. Stogner PDE Verification September 30, 2009 12 / 13

Code Verification Application Verification

Thank you

Roy H. Stogner PDE Verification September 30, 2009 13 / 13