terascale supernova initiative
Post on 23-Feb-2016
44 Views
Preview:
DESCRIPTION
TRANSCRIPT
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
1
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
2
www.tsi-scidac.orgTeraScale Supernova InitiativeTeraScale Supernova Initiative
11 Institution, 21 Investigator, 34 Person, Interdisciplinary Effort ascertain the core collapse supernova mechanism(s) understand supernova phenomenology
e.g.: (1) element synthesis, (2) neutrino, gravitational wave, and gamma ray signatures provide theoretical foundation in support of OS experimental facilities (RHIC, RIA, NUSEL, …) develop enabling technologies of relevance to many applications
e.g. 3D, multifrequency, precision radiation transport serve as computational science testbed
drive development of technologies in simulation “pipeline” (data management, networking, data analysis, and visualization)
Explosions ofExplosions ofMassive StarsMassive Stars Relevance:Relevance:
Element ProductionElement ProductionCosmic LaboratoriesCosmic LaboratoriesDriving ApplicationDriving Application
With ISIC and other collaborators:With ISIC and other collaborators:89 people from 28 institutions involved.89 people from 28 institutions involved.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
3
Components of a Supernova Model1. Accurate (Boltzmann) Neutrino Transport2. Turbulent Stellar Core Flow3. Stellar Core Magnetic Fields4. Gravity (Einsteinian)5. Nuclear (Stellar Core) and Weak Interaction (Neutrino) Physics
Core collapse supernovae are radiatively (neutrino) driven. Fluid instabilities, rotation, and magnetic fields will play a role. Strong (Einsteinian) rather than weak (Newtonian) gravity.
Bruenn, DeNisco, and Mezzacappa (2001)
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
4
TSI’s 3 Paths to 2D/3D Supernova Models
1) 2D/3D Hydrodynamics/MHD with Radial Ray TransportUse existing 1D codes along each ray.Transport parallelized trivially (one ray per processor).
2) 2D/3D Hydrodynamics/MHD with 2D/3D Multigroup Flux-Limited Diffusion (MGFLD)Fully 2D/3D.Sophisticated approximation to Boltzmann transport.Less computationally intensive than Boltzmann transport.Access to 3D science on TeraScale (versus PetaScale) platforms.
3) 2D/3D Hydrodynamics/MHD with 2D/3D Boltzmann TransportFinal word.Explosion mechanism extremely sensitive to transport treatment.MGFLD compares well with Boltzmann in 1D. What about 2D? 3D?
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
5
• TSI members first to perform 1D models with Boltzmann transport.Mezzacappa and Bruenn (1993)Mezzacappa et al. (2001)
• Progress on 2D (and 3D) Boltzmann transport progressing rapidly.
Development of formalism for conservative (energy and lepton number) general relativistic
neutrino transport (analytical tour de force).Cardall and Mezzacappa (2003)
Development of finite differencing. Construction of GenASiS. Completion of test problems. Initiation of realistic 2D supernova studies.
1. Difficult to develop number- and energy- conservative differencing for these “observer corrections (aberration, frequency shift).”2. Difficult to handle “advection” terms when neutrinos and matter are in equilibrium.3. Memory and CPU requirements.
What makes neutrino transport difficult?
Without this, supernova simulationsdifficult to interpret.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
6
Development of radiation field stationary state in nonspherical fixed medium:
Density Distribution Radiation Field Energy Density and Flux
2D Boltzmann Neutrino Transport Test Problem
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
7
Solve for first moment of neutrino distribution(truncation of 2N-1 moments obtained with Boltzmann solution).
2D MGFLD Equations
Advection Terms
Observer Corrections
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
8
First 2D simulations with multifrequency neutrino transport, advection terms, and observer corrections:
Scientific Target: Development of Proto-Neutron Star Instabilities Close coupling of matter and neutrinos requires fully 2D transport for an accurate assessment. What impact do the neutrinos have on the development of these instabilities?
Swesty and Myra (2004)
Running on 1024 processors at NERSC.
Scaling now to 2048.
Fully implicit solve.QuickTime™ and aPlanar RGB decompressor
are needed to see this picture.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
9
Initial shock location/strength depend on knowledge of nuclearstates and their occupationduring core collapse.
This is a challenge in nuclear computation being addressed by TSI’s nuclear theorists.
This challenge is exacerbated by the fact that nuclei increase in size (neutron and proton number)/complexity (population of states, collective excitations) during collapse.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
10
Significant change in initial shock location and strength and stellar core profiles when state of stellar core nuclei computed with more realistic nuclear models and when this new nuclear physics is included in the supernova models.
Hix et al. 2003, Physical Review Letters, 91, 201102.Langanke et al. 2003, Physical Review Letters, 90, 241102.
Merger of two fields at their respective states of the art. (SciDAC enabled.)
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
11
In addition to physics of nuclei during collapse, supernova mechanism depends on high-density physics of “nuclear matter.”
Complex nuclear many-body problem. How sensitive is the supernova mechanism to changes in this physics?
TSI using several “equations of state” (EOSs).1. Lattimer-Swesty (LS, Industry Standard)2. Wilson3. Stone-Newton (SN, New)
LS and SN EOSs are both phenomenological at some level, but fundamentally different.
Will allow us to explore sensitivities to high-densityEOSs despite uncertainties.
Nuclear Matter
Nuclei
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
12
2 fundamentally new instabilities discovered by TSI (computationally):
Stationary Accretion Shock Instability (SASI)
Lepto-Entropy Fingers
Supernova shock wave may become unstable.Instability will
help drive explosion, define explosion’s shape.
Operates between the proto-neutron star andsupernova shock wave.
Blondin, Mezzacappa, and DeMarino (2003)
Operates in the proto-neutron star.Instability may help boost neutrino luminosities,which power the explosion.
Bruenn, Raley, and Mezzacappa (2004)
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
13
QuickTime™ and aAnimation decompressor
are needed to see this picture.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
14
r-process Element Synthesis• Responsible for half the elements heavier than iron.
• Believed to occur above proto-neutron star after explosion in a neutron-rich, neutrino-driven wind.• In past, difficult to obtain “right” conditions in supernova simulations.
Neutrinos driving wind also drive it towardproton richness.
TSI Discovery: An r-process can occur under such conditions…
Near “Symmetric” (equalnumbers of protons andneutrons)
…for rapid wind velocitiesMeyer 2002, Physical Review Letters, 89, 231101
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
15
Implicit Time Differencing…• Extremely Short Neutrino-Matter Coupling Time Scales• Neutrino-Matter Equilibration• Neutrino Transport Time Scales
…Leads to Nonlinear Algebraic Equations Linearize Solve via Multi-D Newton-Raphson Method
Solve Large Sparse Linear Systems
Tera- to Peta-Scale Systems
Correspondence between structure of integro-PDE and underlying linear systems...
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
16
Progress (in conjunction with TOPS):
2D/3D MGFLD
Sparse Approximate Inverse PreconditionerSaylor, Smolarski, and Swesty (2004)
Successfully implemented in 2D MGFLD code (V2D).
2D/3D Boltzmann Transport
“ADI” Preconditioner D’Azevedo et al. (2004)
Successfully implemented in 1D Boltzmann code (AGILE-BOLTZTRAN).• Dense LU factorization was used for dense blocks (D’Azevedo).
Being implemented in 2D/3D Boltzmann code (GenASiS).• Sparse incomplete LU factorization for dense blocks (D’Azevedo, Eijkhout).
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
17
Hydrodynamics VH-1 (PPM) ZEPHYR (Second Order Finite Difference) Zeus-MP (Second Order Finite Difference)
Magnetohydrodynamics Zeus-MP (Second Order Finite Difference)
Neutrino Transport MGFLD_TRAN: 1D General Relativistic Hydrodynamics
with 1D General Relativistic Multifrequency Flux-Limited Diffusion AGILE-BOLTZTRAN: 1D General Relativistic Adaptive Mesh Hydrodynamics
with 1D General Relativistic Boltzmann Transport V2D/V3D: 2D/3D MGFLD Transport Code (Under Development) GenASiS: 2D/3D Boltzmann Code (Under Development)
TSI Radiation Magnetohydrodynamics Code Suite
Extensively analyzed by PERC.Analysis by PERC recently begun.
Zeus-MP Single-CPU performance boosted by factor of 2. Highest stat ever seen at NERSC (30% of peak).
AGILE-BOLTZTRAN Parallel port, scalable linear solve, PERC analysis reduced run times from weeks to days.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
18
InputData
HighlyParallelCompute
Output~500x500files
Aggregate to ~500 files (< 2.2GB each)
HPSSarchive
Data Depot
Logistic NetworkL-Bone
Local MassStorage 14+TB)
Aggregateto one file (~1 TB each)
VizWall
Viz Client
Local 44 Proc.Data Cluster- data sits on local nodes for weeks
Viz Software
As TSI enters “production mode” managing its Workflows has become a paramount issue.
Ideally, we would like to automate these workflows.
Data ManagementNetworkingVisualization
These must be viewed together.
Collaboration between:
SDM Arie Shoshani Nagiza Samatova Guru Kora Ian Watkins Mladen Vouk
Networking Beck Atchley Moore Rao
Visualization (TSI) Blondin Toedte
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
19
Addressing Bulk Data Transfer Needs:Addressing Bulk Data Transfer Needs:
Logistical Networking Light Weight Low Level Deployable … Solution
New Paradigm Integrate storage and networking. Multi-source, multi-stream. Easy for TSI members to share data.
QuickTime™ and aAnimation decompressor
are needed to see this picture.
Atchley, Beck, and Moore (2003)
Data transfer rates 200-300 Mbps using TCP/IP!Limit set by ORNL firewall.Greater rates expected
outside firewall,other protocols (e.g., Sabul).
Direct impact on TSI’s workflow!
Current Data Generation Rate:500 Mbps (10 TB in 2 days).
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
20
CERN
Chicago
Sunnyvale
Atlanta
ANLFNAL
ORNL
CalTech
SLAC
LBL
NERSC
PNNL
10 Gbps
10 Gbps
DOE Science UltraNet + NSF CHEETAH
Seattle
BNL
JLab
University
DOE National Lab
Future Connections
UltraNetCHEETAH
UVa
NCSU
CUNY
TSI serving as a testbed for an NSF-funded network (Cheetah) designed to develop provisioning technologies for dedicatedchannels.
PI: Rao
TSI serving as a testbed for a proposed effortto bring together UltraNet and Cheetah.
PI: Rao
TSI is testbed for a proposed effort to allow a broad research community to access and use UltraNet.
PI: Beck
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
21
QuickTime™ and aYUV420 codec decompressor
are needed to see this picture.
New Rendering Techniques/Quantitative, Comparative Visualization [Kwan-Liu Ma (UC Davis), Pat McCormick, Jim Ahrens (LANL)]
Rapid rendering. Rendering in hardware. Interactivity.
High-resolution, high-quality rendering. Quantitative visualization.
• Multiple variables, gradients and functions of variables.• Computation done in hardware.• Interactivity. Kwan-Liu Ma (UC Davis)
TSI members played a major role in development of Exploratorium/Powerwall at ORNL. Successfully deployed EnSight for
production, remote, and collaborative visualization.
• Development of representations for higher dimensional data for 2D supernova models. Close coupling of
data management, networking, and visualization.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
22
Kwan-Liu Ma (UC Davis)
QuickTime™ and aYUV420 codec decompressor
are needed to see this picture.
04/22/23 Anthony MezzacappaSciDAC PI Meeting, Charleston, SC
23
A number of scientific discoveries have been made in TSI’s first two years of operation.A number of technical and technological breakthroughs have enabled our science.
Execution of the next major stage in our science has begun: Delivery of 2D supernova models.Continue to serve as a testbed for enabling technologies of relevance to other SciDAC applications.
top related