Maria Grazia Pia, INFN Genova Geant4 Physics Validation (mostly electromagnetic, but also...

Maria Grazia Pia, INFN Genova

Geant4 Physics ValidationGeant4 Physics Validation(mostly electromagnetic, but also hadronic…)(mostly electromagnetic, but also hadronic…)

K. Amako, S. Guatelli, V. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen, L. Pandola, S. Parlati, M.G. Pia, T. Sasaki, L. Urban

et al.

Geant4 Space User Workshop

Leuven, 5-7 October 2005


Geant4 Physics ModelsGeant4 Physics Models

Ample variety of physics models in the Geant4 Toolkit– complementary and alternative

Electromagnetic physics – Standard, LowEnergy, Muon, Optical

Hadronic physics– data-driven, parameterised and theory-driven models

Geant4 Physics BookGeant4 Physics Book– on-going project to document the performance of Geant4 physics

against experimental data and in relevant experimental application domains


Validation processValidation process

Geant4 test process– Physics packages are subject to unit and system testing– Verification, validation of single processes/models performed by Working Groups

Validation process– systematicsystematic: cover all models of a given process– comparison to experimental dataexperimental data and established reference databases– rigorous software processrigorous software process to guarantee quality and reliability

– statistical analysisstatistical analysis: quantitative mathematical evaluation

Goals– evaluate quantitatively the accuracy Geant4 physics models– document their respective strength– provide guidance to users to select the models to use in their applications


G.A.P Cirrone, S. Donadio, S. Guatelli, A. Mantero, B. Mascialino, S. Parlati, M.G. Pia, A. Pfeiffer, A. Ribon, P. Viarengo

“A Goodness-of-Fit Statistical Toolkit”IEEE- Transactions on Nuclear Science (2004), 51 (5): 2056-2063

Partly funded by ESA (SEPTIMESS Project)


GoF algorithms GoF algorithms (currently (currently implemented)implemented)

Algorithms for binned distributions – Anderson-Darling test– Chi-squared test – Fisz-Cramer-von Mises test– Tiku test (Cramer-von Mises test in chi-squared approximation)

Algorithms for unbinned distributions – Anderson-Darling test– Cramer-von Mises test– Goodman test (Kolmogorov-Smirnov test in chi-squared approximation)– Kolmogorov-Smirnov test– Kuiper test– Tiku test (Cramer-von Mises test in chi-squared approximation)

In progress– Watson test– Girone Test– Weighted Kolmogorov-Smirnov test– Weighted Cramer-von Mises test

The most complete GoF software system on the market – even among commercial/professional statistics software products)


Overview of recent validation Overview of recent validation activitiesactivities

Geant4 Physics Book: Electromagnetic Volume– comparison against the NIST databases– K. Amako, S. Guatelli, V. N. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen,

L. Pandola, S. Parlati, M. G. Pia, M. Piergentili, T. Sasaki, L. Urban

Comparison of Geant4 electromagnetic physics models against the NIST reference dataIEEE Trans. Nucl. Sci., Vol. 52, Issue 4, Aug. 2005, 910-918

Current Physics Book projects (preliminary results)– Bremsstrahlung final state– Atomic relaxation and PIXE– Bragg peak– Radioactivity from rocks and sands

Other Geant4 validation activities– LCG Simulation Validation Project: focus on hadronic physics

see A. Ribons’s talk at EPS-HEP, Lisbon, July 2005

– Validation of specific physics models done by each Geant4 Working Groups


NIST TestNIST TestPhoton Mass Attenuation Coefficient

Photon Partial Interaction Coefficient – related to the cross section of a specific photon

interaction process

Electron CSDA range and Stopping Power

Proton CSDA range and Stopping Power

CSDA range and Stopping Power

Elements Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb, U

(span the periodic element table)

Energy rangephoton 1 keV – 100 GeVelectron 10 keV – 1 GeV proton 1 keV – 10 GeV 1 keV – 1 GeV

Geant4 models: electrons and photonsStandard

Low Energy EEDL/EPDLLow Energy Penelope

Geant4 models: protons and Standard

Low Energy ICRU49Low Energy Ziegler 1977Low Energy Ziegler 1985Low Energy Ziegler 2000(Low Energy: free electron gas + parameterisations + Bethe-Bloch)

Simulation configuration reproducing NIST conditions (ionisation potential, fluctuations, production of secondaries etc.)


H0: Geant4 simulation = NIST dataH1: Geant4 simulation ≠ NIST data

Statistical analysisStatistical analysis Goodness-of-Fit test (Statistical Toolkit)

GoF test(χ2 test)

Distance between Geant4 simulation

and NIST reference data

Test resultp-value

Geant4 simulation

results+

ReferenceData

The p-value represents the probability that the test statistics has a value at leastat least as extreme as the one observed, assuming the null hypothesis is true

0 0 ≤ p ≤ 1≤ p ≤ 1

p < 0.05p < 0.05 Geant4 simulation and NIST data differ significantly p > 0.05p > 0.05 Geant4 simulation and NIST data do not differ significantly

Alternative hypotheses under test:


Photon mass attenuation coefficientPhoton mass attenuation coefficientGeant4 models:

• Standard• Low Energy – EPDL• Low Energy – Penelope

Reference data: NIST - XCOM

Mass attenuation coefficient in Fe

Geant4 LowE Penelope Geant4 StandardGeant4 LowE EPDLNIST - XCOM

ResultsResults

All Geant4 models compatible with NIST

Best agreement: Geant4 LowE models

H0 REJECTION AREA

Monochromaticphoton beam (Io)

Transmitted photons (I)

0

ln1

I

I

d

p-value stability study

Experimental set-up


Compton interaction coefficient Compton interaction coefficient (cross (cross section)section)


Compton interaction coefficient in Ag

cc )(

AVN

A


H0 REJECTION AREA

Geant4 models:• Standard• Low Energy – EPDL• Low Energy – Penelope


ResultsResults


Best agreement: Geant4 LowE-EPDL


Photoelectric interaction coefficient (cross section)

Geant4 LowE PenelopeGeant4 LowE Penelope Geant4 StandardGeant4 StandardGeant4 LowE EPDLGeant4 LowE EPDLNIST - XCOMNIST - XCOM


Photoelectric interaction coefficient in Ge

phph )(

AVN

A

H0 REJECTION AREA




ResultsResults


Best agreement: Geant4 LowE models


Pair production interaction coefficient Pair production interaction coefficient (cross section)(cross section)


Pair production interaction coefficient in Au

pppp )(

AVN

A


H0 REJECTION AREA

p-va

lue

(pai

r pr

oduc

tion

inte

ract

ion

coef

fici

ent t

est)



ResultsResults

All Geant4 models compatible with NIST and equivalent


Rayleigh interaction coefficient (cross section)Rayleigh interaction coefficient (cross section)

ResultsResultsThe Geant4 Low Energy models look in disagreement with the reference data for some materials

Geant4 LowE Penelope Geant4 LowE EPDLNIST - XCOM

Rayleigh interaction coefficient in Be

rr )(

AVN

A

Be 0.99 1

Al 0.32 <0.05

Si 0.77 <0.05

Fe 1 <0.05

Ge <0.05 0.39

Ag 0.36 0.08

Cs <0.05 <0.05

Au <0.05 <0.05

Pb <0.05 <0.05

U <0.05 <0.05

EPDLXCOM

PenelopeXCOM

Geant4 models:• Low Energy – EPDL• Low Energy – Penelope• (no standard Rayleigh process)



Zaidi H., 2000, Comparative evaluation of photon cross section libraries for materials of interest in PET Monte Carlo simulation IEEE Transaction on Nuclear Science 47 2722-35

The disagreement is evident between 1 keV and 1 MeV photon energies

For what concerns the Geant4 Low Energy EPDL model, the effect observed derives from an intrinsic inconsistency between Rayleigh

cross section data in NIST-XCOM and the cross sections of EPDL97, on which the model

is based

Differences between EPDL97 and NIST-XCOM have already been highlighted in a paper by Zaidi,

which recommends the Livermore photon and electron data libraries as the most up-to-date and

accurate databases available for Monte Carlo modeling.

Rayleigh interaction coefficientRayleigh interaction coefficient

EPDL 97

NIST

Rayleigh interaction coefficient in Au

rr )(

AVN

A


Electron Stopping PowerElectron Stopping PowerExperimental set-up


H0 REJECTION AREA

)(1

SP dx

dE

Geant4 LowE Penelope Geant4 StandardGeant4 LowE LivermoreNIST - ESTAR

Electrons are generated with random direction at the center of the box and stop inside the box

Maximum step allowed in tracking particles was set about1/10 of the expected range

value, to ensure the accuracy of the calculation

Geant4 models:• Standard• Low Energy – EEDL• Low Energy – Penelope

Reference data: NIST – ESTAR (ICRU 37)

ResultsResults

All Geant4 models compatible with NISTand equivalent


Electron CSDA RangeElectron CSDA Range CSDA: particle range without energyloss fluctuations and multiple scattering

Geant4 LowE Penelope Geant4 StandardGeant4 LowE LivermoreNIST - ESTAR

CSDA range in U p-value stability study

H0 REJECTION AREA

Geant4 models:• Standard• Low Energy – EEDL• Low Energy – Penelope

Reference data: NIST – ESTAR (ICRU 37)

ResultsResults

All Geant4 models compatible with NISTand equivalent


Proton stopping power - rangeProton stopping power - rangeStopping power in Al

Geant4 LowE Ziegler 1985Geant4 LowE Ziegler 2000Geant4 StandardGeant4 LowE ICRU 49NIST - PSTAR

+

H0 REJECTION AREA

Stopping power: p-value stability study

H0 REJECTION AREA

CSDA range: p-value stability study

ResultsResultsZiegler parameterisations are as authoritative as ICRU 49 onesComparison rather than validation


stopping power and rangestopping power and range

H0 REJECTION AREA

Stopping power: p-value stability studyCSDA range in Si

Geant4 LowE Ziegler 1977 Geant4 StandardGeant4 LowE ICRU 49NIST - ASTAR

The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the main motivations for the developing of this package.

The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the

main motivations for developing this package


BremsstrahlunBremsstrahlungg

3 sets of models:Standard: G4eBremsstrahlungLow Energy EEDL: G4LowEnergyBremsstrahlungLow Energy Penelope: G4PenelopeBremsstrahlung

3 angular distributions: Tsai, 2BS, 2BN

Angular distribution of photons is strongly model-dependent

Penelope

Standard

Low Energy EEDL (default)

PenelopeLowE-EEDL TSAI (def)

LOWE-EEDL 2BSLOWE-EEDL 2BN

Angle (deg) Angle (deg)


Reference dataReference data

The absolute Bremsstrahlung cross section can be tested

Transmitted energy spectrum at two different

emission angles for four materials (Al, Pb, W,

Ag)

Absolute yields are reported (=

photons/primary), though with an “odd” normalization

R. Ambrose et al., Nucl. Instr. Meth. B 56/57 (1991)

327


Relative comparison...Relative comparison...

Relative comparison (45 degree direction)Shapes of the spectra are in good agreement

Work in progress, will be published

Inte

nsi

ty/Z

(eV

/sr

keV

)

Photon energy (keV)

Photon energy (keV)

LowE-Penelope

Low E EEDL - TSAI

Inte

nsi

ty/Z

(eV

/sr

keV

)


Proton Bragg PeakProton Bragg Peak

Geant4 models: electromagneticGeant4 models: electromagneticStandardLow Energy ICRU 49Low Energy Ziegler 1977Low Energy Ziegler 1985Low Energy Ziegler 2000

Geant4 models: hadronicGeant4 models: hadronicPrecompound + default de-excitationPrecompound + GEM evaporationwith/without Fermi Break-upBinary Cascade

(including Precompound + de-excitation)

Bertini CascadeBertini Cascade + Bertini elastic scattering (when available)ParameterisedGeant4 “educated guess” Medical Dosimetry Physics List

Reference data from CATANA (INFN-LNS Hadrontherapy Group)

Systematic test in progress

Lot of work…

Preliminary results


EM only – StandardEM only – Standard


EM only – ICRU49EM only – ICRU49ENTIRE

PEAK

Exp G4

S 2.89 2.43

T 3.26 3.83

GoF test CVM-AD

ENTIRE

PEAK

Exp G4

S 2.89 2.43

T 3.26 3.83

GoF test CVM-AD

LEFT TAILx<=30mm

Exp G4

S 9.77 11.89

T 2.66 3.16

GoF test CVM-AD

RIGHT TAIL

X>30mm

Exp G4

S 3.89 12.24

T 1.03 1.00

GoF test KS-AD


EM only – ICRU49 – GoF resultsEM only – ICRU49 – GoF resultsALL (N1=149 N2=67) CVM AD

Test statistics 0.112938 0.853737

p-value 0.525095 0.443831

LEFT TAIL x<=30mm

(N1=140 N2=61)

CVM AD


p-value 0.750593 0.606120

RIGHT TAIL X>30mm

(N1=9 N2=6)

KS AD


p-value 0.724871 0.469251


LowE + precompound defaultLowE + precompound defaultALL Exp G4

S 2.89

T 3.26

GoF test CVM-AD

LEFT TAILx<=30mm

Exp G4

S 9.77

T 2.66

GoF test AD

RIGHT TAIL

X>30mm

Exp G4

S 3.89

T 1.03

GoF test KS-AD


ICRU49 + precompound – GoF ICRU49 + precompound – GoF resultsresults

ENTIRE

PEAK

(N1=149 N2=66)

CVM AD


p-value 0.79 0.747452

LEFT TAIL

x<=30mm

(N1=140 N2=60)

CVM AD


p-value 0.97 0.978972

RIGHT TAIL

X>30mm

(N1=9 N2=6)

KS AD


p-value 0.73 0.413129

0.525095 0.443831

ICRU 49 only

0.750593 0.606120

0.724871 0.469251


Nuclear de-excitation alternative modelsNuclear de-excitation alternative models

Work in progress, more to come…


Studies of environmental radioactivity from rocks and sands at the Gran Sasso Laboratory

Geant4 (LowE EM) can the results of a calibration with a 60Co source (in the presence of the sample) reproduce very well

detectorsample

source

simulation

datasimulation

data

Radioactive spectrumRadioactive spectrum

Lower part of the histogram is not meaningful

Anderson-Darling testAnderson-Darling test (for binned data)

Lower E peak– A2 = 0.45 – p-value = 0.80

Higher E peak– A2 = 1.05– p-value = 0.33

Both peaks– A2 = 0.80 – p-value = 0.48



Backscattering coefficient – E=100keVBackscattering coefficient – E=100keV

G4 LowE

Lockwood et al. (1981)

Angle of incidence (withrespect to the normal to the sample surface) = 0°


……and much moreand much more

No time to present all the on-going activities…– atomic relaxation– LCG Simulation Validation project– validation for specific use cases (medical dosimetry, radiation damage to

components, underground experiments etc.)– …

Physics validation is a large component of Geant4 Collaboration’s activities

– Geant4 Physics Validation Workshop, Genova, July 2005– http://www.ge.infn.it/geant4/events/july2005/


Geant4 validation is not an easy Geant4 validation is not an easy job…job…

experimental data often exhibit large differences!


ConclusionConclusion

More results available, no time to show them all…

Systematic, quantitative validation of Geant4 physics in progress– all available models– rigorous statistical analysis

A lot of work!– first paper published– several on-going projects– limited resources available

The validation work provides valauable feedback for the improvement of Geant4 physics models


Communication & PublicationsCommunication & Publications

Feedback from Geant4 users is a very helpful contribution to Geant4 validation

– if you have a problem with Geant4 physics, please tell us– if you get nice results, please tell us too…

IEEE Trans. Nucl. Sci.– major scientific journal about nuclear technologies and instrumentation– many Geant4-related papers published or currently in the review process– M.G. Pia Associate Editor (software - Instrumentation)– please consider publishing your results concerning Geant4 applications

Date post:	15-Jan-2016
Category:	Documents
Upload:	zachariah-freckleton
View:	216 times
Download:	0 times

Maria Grazia Pia, INFN Genova Geant4 Physics Validation (mostly electromagnetic, but also...

Documents