http://www.phy.ornl.gov/tsi/

TeraScale Supernova InitiativeTeraScale Supernova Initiative

Investigator Team

Radiation Transport/Radiation Hydrodynamics

Blondin (NC State) Bruenn (FAU) Hayes (UCSD) Mezzacappa (ORNL) Swesty (SUNYSB)

Nuclear Structure Computationsfor EOS and Neutrino-Nucleus/Nucleon Interactions

Dean (ORNL, UT) Fuller (UCSD) Haxton (INT, Washington) Lattimer (SUNYSB) Prakash (SUNYSB) Strayer (ORNL, UT)

Linear System/Eigenvalue Problem Solution Algorithms for Radiation Transport and Nuclear Structure Computation

Dongarra (UT, ORNL) Saied (UIUC, NCSA) Saylor (UIUC, NCSA)

Visualization

Baker (NCSA) Toedte (ORNL)

Cross-Cutting Team Long-Term CollaborationsStructured like SciDAC

Supernova Science

Blondin Bruenn Fuller Haxton Hayes Lattimer Meyer (Clemson) Mezzacappa Swesty

TOPS

TOPS

SDM

CCAPERCTSTT

GoalGoal Ascertain the explosion mechanism(s). Reproduce supernova phenomenology (element synthesis; neutrino, gravitational wave, and gamma ray signatures; neutron star kicks; gamma ray burst connection)

RelevanceRelevance Dominant source of many elements in the Universe. Given sufficiently well developed models, serve as laboratories for fundamental nuclear and particle physics that cannot be explored in terrestrial laboratories. Driving application in computational science (radiation transport, hydrodynamics, nuclear physics, applied mathematics, computer science, visualization).

ParadigmParadigm Result from stellar core collapse and bounce in massive stars. Radiatively driven (perhaps some are MHD driven, or both).

Need Boltzmann SolutionNeed Angular DistributionNeed Spectrum

“Gray” Schemes InadequateSpectrum ImposedLimited Angular Information (Few Moments)Parameterized (No First Principle Solution)

The bar is high! (10% effects can make or break explosions.)

Janka and Mueller

Herant et al.

Mezzacappa et al.

Fryer and Heger

Burrows, Hayes, and Fryxell

Swesty TSITSIYear 1Year 1

TSITSIYear 2Year 2

TSITSIYear 3Year 3

TSITSIYear 2Year 2

Gra

y M

od

els

Gra

y M

od

els

Sp

ace

Sp

ace

Neutrino EnergyNeutrino Energy

Past Transport in 2D ModelsD: DiffusionFLD: Flux-Limited DiffusionMGFLD: Multigroup FLDMGBT: Boltzmann Transport

Latest TSI 2D/3D Models:

Hydrodynamics only.Focused on understanding 2D/3D flow and its coupling to shock wave.Convectively stable.

2D model exhibits bipolar explosion (due to nonlinear flow-shock interaction).

3D model exhibits similar “long-wavelength” behavior. Key finding.New “rolling” flows identified.

AAS Meeting; Ap.J. SubmittedAAS Meeting; Ap.J. Submitted

2D Model2D Model

3D Model3D Model

What is the Recipe for Explosion?

Are there multiple mechanisms? Neutrino-driven supernovae MHD-driven supernovae Supernovae driven by both neutrinos and MHD effects

One mechanism for a class of stars?

Is the mechanism tailored to the individual star?

Neutrino Heating

Convection

Rotation

Magnetic Fields

GeneralRelativity

Thomas Fermi (Classical)

Hartree-Fock

Shell Model DiagonalizationShell Model Monte Carlo

Bloch-Horowitz

Advanced solutions to the many body problem.

Solve “exact” many-body problem.

Time

Classical treatment of many-bodyproblem.

Lowest order solution to the quantum mechanical many-body problem.

-nucleus

High-Density EoS

Ensembles

-decay

e-capture

Nuclear MatterOpacities

Densityof States

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

R-Process Breakthrough

r-process can occur in “symmetric” environment (equal numbers of protons and neutrons) under certain conditions (high entropy, fast expansion).

Meyer, PRL Submitted

HydrodynamicsExplicit Differencing

Reactive FlowsNewtonian

General Relativistic

RadiationTransport

Implicit DifferencingMGFLD

PreconditionersSparse System Solvers

MGBTPreconditioners

Sparse System Solvers(Matrix Free)

Nuclear,Weak Interaction

PhysicsThermodynamics

(Composition),Neutrino Sources and Interactions

Supernova Science

Generation 1

Generation 2

Generation 3

Integration of Technologies

2D MGFLD Simulation with NaiveNeutrino Interactions and Single-Nucleus Equation of State

Computation of State of the Art Neutrino-Matter Interactions

Inclusion of state of the art neutrino interactions in “Generation 1” MGBT/MGFLD Simulations

High-Resolution 3D MGFLD with Full Integration of Components (Ensemble of Nuclei, State of the Art Neutrino-Matter Interactions, ...)

Supernova Simulation TimelineSupernova Simulation Timeline

Year 0 Year 1 Year 2 Year 3 Year 4 Year 5

3D MGFLD Models (4D)

3D MGFLD Models w/ AMR (4D)2D MGFLD Models (3D)

2D Boltzmann Models (5D)

1D Boltzmann Models (3D) 3D Boltzmann Models (6D)

Implicit Time DifferencingExtremely Short Neutrino-Matter Coupling Time ScalesNeutrino-Matter EquilibrationNeutrino Transport Time Scales

Nonlinear Algebraic EquationsLinearizeSolve via Multi-D Newton-Raphson Method

Large Sparse Linear Systems

ISIC Collaborations: TOPSISIC Collaborations: TOPS

Progress:Sparse Approximate Inverses for 2D MGFLD (Saylor, Smolarski, Swesty; J. Comp. Phys.)ADI-Like Preconditioner for Boltzmann Transport (D’Azevedo et al.; Precond 2001, NLAA)

AGILE-BOLTZRAN, V2D codes turned over to TOPS for analysis and development.

Boltzmann Equation nonlinearintegro-PDE

Memory Requirements (assuming matrix-free methods): 10s Gb up to 1/2 Tb

TSI Code: F90 + MPI Code Object-Oriented Design for Interoperability and Reuse Application Framework:

IBEAM = Interoperability Based Environment for Adaptive Meshes NASA HPCC-Funded Project (PI: Swesty)

AMR: PARAMESH

ISIC Collaborations: CCTTSSISIC Collaborations: CCTTSS

Goal: Develop our framework to be CCA-compliant.Initiated discussions with ANL, LLNL, and ORNL members of CCTTSS.

Assess Code Performance on Parallel Platforms Identify Code Optimizations to Increase Performance

TSI Code SuiteHydrodynamics:

VH-1 (PPM)ZEPHYR (Finite Difference)

Neutrino Transport:AGILE-BOLTZTRAN: 1D General Relativistic Adaptive Mesh

Hydrodynamics with 1D Boltzmann TransportV2D: 2D MGFLD Transport Code V3D: 3D MGFLD Transport Code (Under Development)2D/3D Boltzmann Code (Under Development)

Re

su

lts

fo

r V

H-1

Re

su

lts

fo

r V

H-1

ISIC Collaborations: PERCISIC Collaborations: PERC

VH-1 numerical hydrodynamics algorithmscales well.

ISIC Collaborations: SDMISIC Collaborations: SDM

Use PROBE environment for staging data between simulation platforms and end-user visualization platforms. Develop new data analysis techniques/tools tailored to our application, allowing (a) data reduction and (b) discovery potential. Use of agent technology for distributed data analysis (data analysis must be done in parallel to achieve reasonable throughputs).

Adaptive Quadratures (Direction Cosines) for Multidimensional Radiation Transport

Results for 1D Boltzmann Transporton Milne Problem (D’Azevedo):

Greatest challenge to completing 3D Boltzmann simulations is memory.Minimize number of quadratures to minimize memory needs while maintaining physical resolution. (Also important for 1D/2D MGBT.)Optimization Problem

ISIC Collaborations: TSTTISIC Collaborations: TSTT

Extended Core

Compact Core

Identify Optimal Paths in Our Collaborative Visualization Server-Client ModelMaximize Bandwidth along these Paths (Not Achieved Using Current Protocols)

Collaboration with Supporting Base Projects: NetworkingCollaboration with Supporting Base Projects: Networking

Participated in ORNL Workshop on DoE High-Performance Network R&D and Applications

Convey TSI Needs to Networking Team Participate in White Paper to Define and Develop Interface between Efforts