INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Fusion Status Report
Francisco Castejón ([email protected])
CIEMAT. Madrid, Spain.
Grid Users Forum, CERN, March 2006 2
Enabling Grids for E-sciencE
INFSO-RI-508833
Outline
• Strategy.• Fusion Deployment and VO setup.
– The problem of the name
• Present Applications: Computing in Plasma Physics.• Future Applications in the grid. • Data storage and handling.• Conclusions.
Grid Users Forum, CERN, March 2006 3
Enabling Grids for E-sciencE
INFSO-RI-508833
Strategy• Computing:
– Identify common Codes suitable for GRID. (Ongoing)
– Adapt codes to the GRID. (Ongoing)
– Set up VO (Ongoing)
– Production phase.
Data handling: Define strategies for data storage. & database organization.Protocol for data Access.
Grid Users Forum, CERN, March 2006 4
Enabling Grids for E-sciencE
INFSO-RI-508833
ITER: Making decisions in real Time !!
Data Acquisition and Storage (Grid, Supercomputers)
Data Analysis and Reduction: Artificial Intelligence, Neural Network, Pattern Recognition
Simulations: Large codes in different platforms (Grid, Supercomputers)
Decision for present/next shot
One half an hour shot every one hour and a half: Decisions in real time.
Grid Users Forum, CERN, March 2006 5
Enabling Grids for E-sciencE
INFSO-RI-508833
ITER Partners
Distributed Participation. Distributed Participation. Data access. Remote Control Data access. Remote Control
Rooms?Rooms?
Grid Users Forum, CERN, March 2006 6
Enabling Grids for E-sciencE
INFSO-RI-508833
International Tokamak (ITPA) and Stellarator (SIA) collaborations.
USA:USA:
Alcator C-Mod (MIT)Alcator C-Mod (MIT)
DIII-D (San Diego)DIII-D (San Diego)
NSTX (Princeton)NSTX (Princeton)
NCSX (Princeton)NCSX (Princeton)
HSX (Wisconsin)HSX (Wisconsin)
QPS (Oak-Ridge)QPS (Oak-Ridge)
USA Fusion GridUSA Fusion Grid
Russia:Russia:
T-10 (Kurchatov)T-10 (Kurchatov)
Globus (Ioffe)Globus (Ioffe)
T-11M (TRINITI)T-11M (TRINITI)
L-2 (Gen. Inst. Phys.)L-2 (Gen. Inst. Phys.)
EGEE ProjectEGEE Project
EU:EU:
JET (EFDA)JET (EFDA)
ASDEX (Ger.)ASDEX (Ger.)
TORE SUPRA (Fran.)TORE SUPRA (Fran.)
MAST (UK)MAST (UK)
TEXTOR (Ger.)TEXTOR (Ger.)
TCV (Switz.)TCV (Switz.)
FTU (Italy)FTU (Italy)
W7-X (Ger.)W7-X (Ger.)
TJ-II (Spain)TJ-II (Spain)
EGEE ProjectEGEE Project
Japan:Japan:
JT-60 (Naka)JT-60 (Naka)
LHD (Toki)LHD (Toki)
CHS (Nagoya)CHS (Nagoya)
H-J (Kyoto)H-J (Kyoto)
GRID Project ?GRID Project ?
China, Brazil, Korea, India:China, Brazil, Korea, India:
KSTAR (Korea) KSTAR (Korea)
TCBRA (Bra.) TCBRA (Bra.)
H-7 (China)H-7 (China)
U2A (China)U2A (China)
SST1 (India) SST1 (India)
EGEE ProjectEGEE Project
Grid Users Forum, CERN, March 2006 7
Enabling Grids for E-sciencE
INFSO-RI-508833
PARTNERS and Resources for VO
• SW Federation: CIEMAT, BIFI, UCM, INTA (Spain) • Kurchatov (Russia). • Culham Laboratory- UKAEA (UK) • KISTI (South Korea). • ENEA (Italy).• CEA-Cadarache (France).• …
Experience in using and developing Fusion Applications.Experience in porting applications and developing Grid Technologies.
Connection with EELA (Some fusion partners: Brazil,Mexico,Argentina)
Needed: Join IPP-Max Planck (Germany) and other EFDA Associations. Also needed: contact with USA, China, Japan,…
Grid Users Forum, CERN, March 2006 8
Enabling Grids for E-sciencE
INFSO-RI-508833
VO Deployment
• Present: CIEMAT: 27 KSpecInts; BIFI: 8 KSpecInts; INTA: 6 KSpecInts
Resource Broker in BIFI (Spain) VO Manager: I. Campos (BIFI. Spain)
http://grid.bifi.unizar.es/egee/fusion-vo/
http://www-fusion.ciemat.es/collaboration/egee/
• Within less than 6 months:
JET: 38 KSpecInts; BIFI: 32 KSpecInts; CEA-Cadarache ?
KISTI?, INTA?, ENEA?
• Within less than 6 months:
JET: 38 KSpecInts; BIFI: 32 KSpecInts; CEA-Cadarache ?
KISTI?, INTA?, ENEA?
• Beginning of 2007:
JET: 32 additional cores; BIFI: 32 additional cores; CIEMAT ?;
CEA-Cadarache ?(second phase already committed).
• Beginning of 2007:
JET: 32 additional cores; BIFI: 32 additional cores; CIEMAT ?;
CEA-Cadarache ?(second phase already committed).
Grid Users Forum, CERN, March 2006 9
Enabling Grids for E-sciencE
INFSO-RI-508833
VO Deployment: The problem of the name.
• Russian Grid has adopted the same name of Fusion, as we have done.
• The works that are sent by our resource broker go to such Grid.
• Our VO deployment is hindered.• They should change the name in short term (~1 week). A
suitable name: Fusion-RIDG• Otherwise: We have to change the name. Consequences
on Russian certificates.
Grid Users Forum, CERN, March 2006 10
Enabling Grids for E-sciencE
INFSO-RI-508833
COMPUTING in the GRID: Present Applications
– Applications with distributed calculations: Monte Carlo, Separate estimates, …
– Multiple Ray Tracing: e. g. TRUBA. – Stellarator Optimization: VMEC– Transport and Kinetic Theory: Monte Carlo Codes.
Grid Users Forum, CERN, March 2006 11
Enabling Grids for E-sciencE
INFSO-RI-508833
Multiple Ray Tracing: TRUBA
Single Ray (1 PE): Single Ray (1 PE): Hamiltonian Hamiltonian Ray Tracing Equations.Ray Tracing Equations.
Beam Beam Simulation: Simulation:
Bunch of rays Bunch of rays with beam waist with beam waist far from the far from the critical layer critical layer (100-200 rays)(100-200 rays)
Bunch of rays Bunch of rays with beam waist with beam waist close to the close to the critical layer critical layer (100-200 rays) x (100-200 rays) x (100-200 wave (100-200 wave numbers) numbers) ~10~1055
GRID GRID PROBLEMPROBLEM
Grid Users Forum, CERN, March 2006 12
Enabling Grids for E-sciencE
INFSO-RI-508833
TRUBA: Multiple Ray Tracing
TRUBA for EBW: TRUBA for EBW: -Real geometry in TJ-II:Coming from a supercomputer Real geometry in TJ-II:Coming from a supercomputer (VMEC).(VMEC).
- A single Non-relativistic ray (about 18’).A single Non-relativistic ray (about 18’).- A single relativistic ray (about 40’).A single relativistic ray (about 40’).
- Some problems with Geometry libraries.Some problems with Geometry libraries.
- Ported to the grid using Grid Way (for the moment). Ported to the grid using Grid Way (for the moment). -See:See:J. L. VJ. L. Vázquez-Poleti. “Massive Ray Tracing ázquez-Poleti. “Massive Ray Tracing in Fusion Plasmas on EGEE”. User Forum, in Fusion Plasmas on EGEE”. User Forum, 2006.2006.
Grid Users Forum, CERN, March 2006 13
Enabling Grids for E-sciencE
INFSO-RI-508833
Optimised Stellarators QPS and NCSXSupercomputer Optimization
NCSXQPS
Grid Users Forum, CERN, March 2006 14
Enabling Grids for E-sciencE
INFSO-RI-508833
Stellarator optimization in the Grid
-A lot of different Magnetic Configurations operating nowadays.
OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.
Every variant computed on a separate processor (~10’) VMEC (Variational Momentum Equilibrium Code)
120 Fourier parameters are varied.
-A lot of different Magnetic Configurations operating nowadays.
OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.
Every variant computed on a separate processor (~10’) VMEC (Variational Momentum Equilibrium Code)
120 Fourier parameters are varied.
rB(ψ,θ,ϕ ) =
rBm,n
m,n∑ (ψ )ei(mθ−nϕ )
R(ψ ) = Rm,nm,n∑ (ψ )cos(mθ −nϕ )
Z(ψ ) = Zm,nm,n∑ (ψ )sin(mθ −nϕ )
Plasma configuration may be optimised numerically by variation of the field parameters.
Plasma configuration may be optimised numerically by variation of the field parameters.
Grid Users Forum, CERN, March 2006 15
Enabling Grids for E-sciencE
INFSO-RI-508833
VMEC on Kurchatov GRID
• LCG-2 - based Russian Data Intensive Grid consortium resources.
• About 7.500 cases computed (about 1.500 was not VMEC-computable, i.e. no equilibrium).
• Each case took about 20 minutes. • Up to 70 simultaneous jobs running on the grid.• Genetic Algorith used to select the optimum case.
- See:See:V. Voznesensky. “Genetic Optimisations in Grid”. V. Voznesensky. “Genetic Optimisations in Grid”. User Forum, 2006.User Forum, 2006.
• LCG-2 - based Russian Data Intensive Grid consortium resources.
• About 7.500 cases computed (about 1.500 was not VMEC-computable, i.e. no equilibrium).
• Each case took about 20 minutes. • Up to 70 simultaneous jobs running on the grid.• Genetic Algorith used to select the optimum case.
- See:See:V. Voznesensky. “Genetic Optimisations in Grid”. V. Voznesensky. “Genetic Optimisations in Grid”. User Forum, 2006.User Forum, 2006.
Grid Users Forum, CERN, March 2006 16
Enabling Grids for E-sciencE
INFSO-RI-508833
Kinetic Transport• Following independent particle orbits
• Montecarlo techniques: Particles distributed according to experimental density and ion temperature profiles (Maxwellian distribution function)
• SUITABLE PROBLEM FOR CLUSTER AND GRID TECHNOLOGIES
rVD =
rE ×
rB
B2 +m2q
2v2 −v⊥2( )
rB×∇
rB
B3
Grid Users Forum, CERN, March 2006 17
Enabling Grids for E-sciencE
INFSO-RI-508833
Kinetic Transport
Example of orbit in the real 3D Example of orbit in the real 3D TJ-II Geometry (single PE). TJ-II Geometry (single PE).
~1 GBy data, 24 h x 512 PE~1 GBy data, 24 h x 512 PE
Distribution function of parallel Distribution function of parallel velocity at a given position velocity at a given position (Data Analysis).(Data Analysis).
Grid Users Forum, CERN, March 2006 18
Enabling Grids for E-sciencE
INFSO-RI-508833
Kinetic transport
No collisions: 0.5 ms of trajectory takes 1 sec. CPU..No collisions: 0.5 ms of trajectory takes 1 sec. CPU..
Collisions: 1 ms of trajectory takes 4 sec CPU. Collisions: 1 ms of trajectory takes 4 sec CPU. Particle life: 150 - 200 ms. Single particle Particle life: 150 - 200 ms. Single particle ~ 10 min.~ 10 min.
Necessary statistics for TJ-II 10Necessary statistics for TJ-II 1077 particles. particles.
Grid Users Forum, CERN, March 2006 19
Enabling Grids for E-sciencE
INFSO-RI-508833
COMPUTING in the GRID: Future applications
– EDGE2D Application for tokamaks
– Transport Analysis of multiple shots (typically 104 shots) or Predictive Transport with multiple models: e. g. ASTRA. CIEMAT(Spa) + IPP(Ger) + Kurchatov(Rus) + EFDA(UE) + …
– Neutral Particle Dynamics: EIRENE:CIEMAT(Spa) + IPP(Ger)
Grid Users Forum, CERN, March 2006 20
Enabling Grids for E-sciencE
INFSO-RI-508833
JET – Flagship of Worldwide Fusion: EDGE2D Equilibrium code.
Grid Users Forum, CERN, March 2006 21
Enabling Grids for E-sciencE
INFSO-RI-508833
Cross section of present EU D-shaped tokamaks compared to the ITER project
EDGE2D: Determine plasma shape from Measurements: Plasma current, Pressure,
Magnetic field…
rj ×
rB=
r∇p
-EDGE2D code solves the 2 D fluid equations for the conservation of energy, momentum and particles in the plasma edge region.
-Ions, electrons and all ionisation stages of multiple species are considered.
-Interaction with the vessel walls is simulated by coupling to monte-carlo codes, to provide the neutral ion and impurity sources.
Grid Users Forum, CERN, March 2006 22
Enabling Grids for E-sciencE
INFSO-RI-508833
Massive Transport Calculations
For Instance: Enhanced heat Confinement in TJ-II. Lower heat diffusivity for low electron density and high absorbed power density.
A different case on every PE.
0
0.5
1
1.5
2
-15 -10 -5 0 5 10 15
Pe-2562Pe-2565Pe-2558Pe-2560Pe-2559
<r> (cm)
0
0.5
1
1.5
-15 -10 -5 0 5 10 15
2562256525582560.2559
<r> (cm)0
0.2
0.4
0.6
0.8
1
1.2
-15 -10 -5 0 5 10 15
25592560255825652562
<r> (cm)
0.1
1
10
0 5 10 15
Chi-2562Chi-2565Chi-2558Chi-2560Chi-2559
r (cm)0
1
2
3
4
5
0.5 1 1.5n
0T
e-3/2 (1019 m-3 keV-3/2)
Grid Users Forum, CERN, March 2006 23
Enabling Grids for E-sciencE
INFSO-RI-508833
EIRENEEIRENE CodeCode
Trayectory of a He atom in TJ-II. Vertical and horizontal proyections. It starts in the green point and is absorbed in the plasma by an ionization process. The real 3D geometry of TJ-II vacuum chamber is considerd.
Grid Users Forum, CERN, March 2006 24
Enabling Grids for E-sciencE
INFSO-RI-508833
DATA HANDLING
Storage: Large data flux: 104 sensors x 20-50 kHz sampling= 1-10 GBy per second raw datax 0.5 h= 3 TBy per shot in ITER every 1,5 h
Data Access & Sharing in Large Cooperative Experiments:Strategy to be defined:-Database with distributed access or distributed storage?-The evolution of technologies until ITER works.
If distributed storage: We need a standard representation for experimental data in LCG-2/gLite CE middleware.Storage should allow to do some basic processing: neural network, clustering…
Grid Users Forum, CERN, March 2006 25
Enabling Grids for E-sciencE
INFSO-RI-508833
DAS Tools: Visualization, DAQ and processing
To add grid-aware protocols for:
• Data navigation and mining
• Data exchange
• Data search
• Event catch
Grid Users Forum, CERN, March 2006 26
Enabling Grids for E-sciencE
INFSO-RI-508833
Conclusions
• VO of Fusion grid is almost ready (the problem of the name).
• Effort to get more partners in and outside EFDA is being done.
• Several applications are running in the grid.• Future applications for the grid are identified.• A deep discussion and investigation on large
amount of data handling is needed. • Cook book for data handling in the grid is desirable.