1

V&V: What to Expect When You’re Doing Science

Karen PaoDivision of Advanced Simulation and Computing (ASC)

Office of R&D for National Security S&TNational Nuclear Security Agency (NNSA)

US Dept of EnergyKaren.pao@nnsa.doe.gov

2

You have the opportunity to do things in the right order…

We wished we had better experiments for V&V!

3

The ASC program has a V&V subprogram – a rarity in simulation programs!

4

Where others think the NNSA is today

• Consider the JASON RRW report – even though it never explicitly mentioned V&V:– “…A concern remains, however, that even though codes can reproduce the

performance of previously tested weapons, it is not yet possible to quantify how well excursions from a tested design can be modeled and predicted…”

• This and other concerns led to a new Congressional direction to establish the “Advanced Certification” Campaign to address: – improvement of the weapons certification process through expanded,

independent peer review mechanisms and refinement of computational tools and methods;

– advancement of the physical understanding of surety mechanisms;

– further exploration of failure modes;

– manufacturing process assessments; and

– the study of strategic system-level requirements.

Is Science-Based Stockpile Stewardship a house of cards?

5

You might not see the V&V, but it has to be there

Not to Scale

Code and Model Development

Applications(design/analysis)

V & V is the backbone of all simulations, the pervasive evaluator

6

We have a Vision for V&V

Assess Simulation Credibility

Advise Simulation Community

Advocate Simulation Capability

7

Consequences of simulations

• Important national security decisions will be made based on, ultimately, simulations

– A person will make the decision. What evidence will the person possess?

• Some examples of decisions that would, could, or should be made with help from simulations:

– Annual Assessment: stockpile systems are evaluated to determine whether or not the Nation needs to go back to underground nuclear testing

– Evaluation of the “Preferred Alternative” of NNSA’s “Complex Transformation:” When does the Nation need a particular facility to build particular parts?

– Consequence Management: planning for an urban detonation event – high-fidelity simulations will help planners, first-responders, and decision-makers determine what will be done in such an event

What would it take for someone to believe simulation results – enough to make a decision

based on them?