Long-TermSimulationofBeam-BeamDynamicswithGHOST

BalšaTerzićDepartmentofPhysics,OldDominionUniversity

CenterforAcceleratorStudies(CAS),OldDominionUniversity

OldDominionUniversity:K.Arumugam,R.Majeti,S.Sangam,C.Cotnoir,D.Ranjan,M.Zubair,A.GodunovJeffersonLab:V.Morozov,Y.Roblin,H.Zhang,F.Lin,E.Nissen,T.Satogata

Beam-BeamEffectsinCircularCollidersWorkshop,LBNL,February7,2018

February 7, 2018 Beam-Beam with GHOST 1

Outline

February 7, 2018 Beam-Beam with GHOST 2

• Motivationandchallengesofbeam-beamsimulationsofJLEIC• Computationalrequirements• “Gearchange”schemeforbeamsynchronization

• GHOSTcodedevelopment• Outlineoftheimplementation(trackingandcollision)• Simulating“gearchange”

• GHOSTstatusreport

ComputationalRequirements

February 7, 2018 Beam-Beam with GHOST 3

• Perspective:AtthecurrentlayoutoftheJLEIC1hourofmachineoperationtime≈400millionturns• Requirementsforlong-termbeam-beamsimulationsofJLEIC

① High-ordersymplecticparticletracking② Speed③ Beam-beamcollision④ “Gearchange”forbeamsynchronization

Implicationof“GearChange”

February 7, 2018 Beam-Beam with GHOST 4

•  Beamsynchronization–highlydesirable•  Smallermagnetmovement•  SmallerRFadjustment

•  Detectionandpolarimetry–highlydesirable•  Cancellationofsystematiceffectsassociatedwithbunchcharge

andpolarizationvariation–greatreductionofsystematicerrors,sometimesmoreimportantthanstatistics

•  Simplifiedelectronpolarimetry–onlyneedaveragepolarization,mucheasierthanbunch-by-bunchmeasurement

•  Dynamics?•  Possibilityofaninstability–needstobestudiedfurther

(Hirata&Keil1990;Haoetal.2014)

Fastbeam

Slowbeam

GHOST:Outline

February 7, 2018 Beam-Beam with GHOST 5

• GHOST:Gpu-acceleratedHigh-OrderSymplecticTrackingDesignedanddevelopedfromscratch

• GHOSTresolvescomputationalbottlenecksby:• Usingone-turnmapsforparticletracking• EmployingBassetti-Erskineapproximationforcollisions• Implementingthecodeonamassively-parallelGPUplatform

• WhyGPUs?• Idealfor“sameinstructionformultipledata”(particletracking)• Bestwhennocommunicationrequired(tracking;collision)• Moore’slawstillappliestoGPUs(nolongerforCPUs)

• Twomainparts:1.Particletracking 2.Beamcollisions

GHOST:SymplecticParticleTracking

February 7, 2018 Beam-Beam with GHOST 6

• Symplectictrackingisessentialforlong-termsimulations

SympleticTracking500000turns,3rdordermap

x

EnergyisnotconservedParticlewillsoonbelost

Energyisconserved

Non-SympleticTracking500000turns,3rdordermap

x

pxpx

GHOST:SymplecticParticleTracking

February 7, 2018 Beam-Beam with GHOST 7

• Higher-ordersymplecticityrevealsmoreaboutdynamics

2ndordersymplectic

4thordersymplectic

3rdordersymplectic

5thordersymplectic

5000turns

GHOST:SymplecticParticleTracking

February 7, 2018 Beam-Beam with GHOST 8

• SymplectictrackinginGHOSTisthesameasinCOSYInfinity(Makino&Berz1999)• Startwithaone-turnmap

• Symplecticitycriterionenforcedateachturn

• Involvessolvinganimplicitsetofnon-linearequations• Introducessignificantcomputationaloverhead

x =X

↵��⌘�µ

M(x|↵��⌘�µ)x↵a�y�b⌘l��µ

Initialcoordinates Finalcoordinates

GHOST:SymplecticParticleTracking

February 7, 2018 Beam-Beam with GHOST 9

• SymplectictrackinginGHOSTisthesameasinCOSYInfinity(Makino&Berz1999)

Non-SympleticTracking3rdordermapCOSYGHOST100,000turns

SympleticTracking3rdordermapCOSYGHOST100,000turns

Perfectagreement!

GHOST:SymplecticParticleTracking

February 7, 2018 Beam-Beam with GHOST 10

• Dynamicaperturecomparisontoelegant(Borland2000)• 400millionturnsimulation(trulylong-term)GHOSTelegant1,000turns Sympl.4thorder,400millionturns

Goodagreement

GHOST:BeamCollisions

February 7, 2018 Beam-Beam with GHOST 11

• Bassetti-Erskineapproximation• Beamsas2DtransverseGaussianslices• Poissonequationreducestoacomplexerrorfunction• Finitelengthofbeamssimulatedbyusingmultipleslices

• Wegeneralizeda“weak-strong”formalismofBassetti-Erskine• Include“strong-strong”collisions(eachbeamevolves)• Includevariousbeamshapes(originallyonlyflatbeams)

GHOSTBenchmarking:Collisions

February 7, 2018 Beam-Beam with GHOST 12

• Codecalibrationandbenchmarking• ConvergencewithincreasingnumberofslicesM• ComparisontoBeamBeam3D(Qiang,Ryne&Furman2002)GHOST,1cmbunch40kparticles

GoodagreementwithBeamBeam3D

BeamBeam3D&GHOST,10cmbunch40kparticles

Finitebunchlengthaccuratelyrepresented

GHOSTBenchmarking:HourglassEffect

February 7, 2018 Beam-Beam with GHOST 13

• Whenthebunchlengthσz≈β*attheIP,weobserveageometricreductioninluminosity–thehourglasseffect(Furman1991)

GHOST,128kparticles,10slices

Goodagreementwiththeory

“GearChange”withGHOST:Approach

February 7, 2018 Beam-Beam with GHOST 14

• “Gearchange”providesbeamsynchronizationforJLEIC• Collisionofbeamswithdifferentnumberofbunches(n1,n2)ineachring(JLEIC:n1≈3420)• Ifn1andn2aremutuallyprime,allcombinationsofbunchescollide• Initially:n1=n2+1;moreflexibility:generaln,m

• Brokensymmetry:from1x1tonx(n+1)• ThecomputationalloadisalleviatedwithGPUs• n2collisionshappenconcurrently–highlyparallelizable• informationforallbunchesisstored–largememoryload

• Collidemultiplebunchpairsonapredeterminedschedule• nbunchdifferentpairscollideateachturn–highlyparallelizable

Fastbeam

Slowbeam

“GearChange”withGHOST:nx(n-1)

February 7, 2018 Beam-Beam with GHOST 15

nbunches

OneturnonK40GPUs: t = 0.087nbunches

nGPUssec

Executiontimescalesas1/nGPUs

Executiontimeperturn

“GearChange”withGHOST:nx(n-1)

February 7, 2018 Beam-Beam with GHOST 16

nGPUs>30

t = 0.087nbunches

nGPUssecOneturnonK40GPUs:

Executiontimeperturn

Current/FutureEfforts:OtherFunctionalities

February 7, 2018 Beam-Beam with GHOST 17

GHOST’smodularnatureallowsforflexibilityinsimulations

process

map

GHOST:StatusReport

February 7, 2018 Beam-Beam with GHOST 18

• Stage1:Particletracking• High-order,symplectictrackingoptimizedonGPUs• BenchmarkedagainstCOSY:Exactmatch• 400millionturntracking-onlysimulationcompleted

• Stage2:Beamcollisionsand“gearchange”• Single-bunchcollisionimplementedonmultipleGPUs• nx(n-1)collisionimplementedonamultipleGPU(arbitraryn)• GeneralnxmcollisionimplementedonasingleGPUs(soonmulti-GPU)

• Stage3:Benchmarkingandsimulations(underway)• Multiple-bunchvalidation,checking,benchmarkingandoptimization• SystematicsimulationsofJLEICand“gearchange”• Othercollisionmethods:fastmultipole• Synchrotronradiation,spacecharge,electroncooling,multipleIPs,IBS

þ

þ

þ

þ

þ

þ

February 7, 2018 Beam-Beam with GHOST 19

BackupSlides

BenchmarkingGHOST

February 7, 2018 Beam-Beam with GHOST 20

• BenchmarkingGHOSTwithothercodes(currentlyunderway)1.  SystematicallycomparetoBeamBeam3DandGuineaPig2.  Conductconvergencestudies3.  ReproducethehourglasseffectfortheaggressiveJLEICdesign

• Gearchangesimulations

1.  Simulate“gearchange”effectsforlownumberofbunches(Reproduce11-10asinHaoetal.2014)

2.  ScaleuptofullJLEICparameters(3420/3419,3420/3418,…bunches)

JLEICDesignParametersUsed

February 7, 2018 Beam-Beam with GHOST 21

Speedup6Slices 1Turn

Npart CPU GPU SpeedupCPUTracking Collision Tracking Collision

1000 0.644896 13.2116 0.644768 15.4794 0.8510000 1.02157 129.49 1.04451 17.9388 7.22100000 5.86016 1287.17 5.91194 29.8827 43

1000000 54.5349 12851 54.8268 147.746 86

10kParticles 6SlicesNturn CPU GPU SpeedupCPU

Tracking Collision Tracking Collision1 1.04202 129.479 1.03523 17.823 7.26

10 0.953088 131.204 0.96128 17.7718 7.38

100 0.965376 143.975 0.961472 17.4446 8.25

1000 0.951872 119.376 0.989312 12.4215 9.61

1MillionPatricles 1TurnNslices CPU GPU SpeedupCPU

Tracking Collision Tracking Collision1 54.473 2235.67 54.8347 30.738 722 54.4848 4396.9 54.7464 54.4933 813 54.4546 6480.04 54.7209 75.2644 864 54.4835 8612.99 54.8068 99.6275 865 54.5129 10708.2 54.7883 125.001 866 54.4469 12843.6 54.7732 147.913 87

February 7, 2018 Beam-Beam with GHOST 22