Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | laurel-stanley |
View: | 215 times |
Download: | 0 times |
Tevatron IPM status Tevatron IPM status
Oct 2009Oct 2009
Andreas JanssonK.Bowie, T.Fitzpatrick, R.Kwarciany,
C.Lundberg, D.Slimmer, L.Valerio, J.Zagel
Also thanks to:
T.Anderson, M.Bowden, A.Bross, A. Chen, R.Dysert, S.McCormick, S. Suleimani, H.Nguyen, C.Rivetta, H.Glass, D.Harding, B.Hively, V. Kashikin, D.Miller, Z.Tang, J.Volk, T.Zimmermann, …
Outline
• Brief overview history of Tev IPMs– Summary of Review talk from 2007 – Long version available at:
http://indico.fnal.gov/conferenceDisplay.py?confId=1272
• What happened then?– Progress reports from 2007 to now– Summary of partial reports given at Tev
Dept & Vaia’s meetings
• Current status
10/15/2009 IPM status October 2009
DESIGNBrief overview of IPM history
10/15/2009 IPM status October 2009
9/21/07 IPM Review, September 21, 2007 A. Jansson 4
Goal
GOAL:Measure protons and pbar beam size
turn by turn at injection and ramp during normal operation to diagnose
and mitigate emittance blow-up.
9/21/07 IPM Review, September 21, 2007 A. Jansson 5
Challenges in the Tevatron
• Two small beams separated by helix.
• Separate protons from pbars, injected from circulating beam
• Beam induced parasitic signals.
• Low vacuum pressure
• Fine granularity and many channels
• Single bunch resolution and gating
• Improved shielding and matched cables
• Local pressure bump with controlled leak
Solutions:Challenge:
9/21/07 IPM Review, September 21, 2007 A. Jansson 6
IPM detector
0.2T
MCP1kV100V
0V
0V
9.9kV10kV
All signal cables are enclosed in a Faraday cage!
9/21/07 IPM Review, September 21, 2007 A. Jansson 7
Anode board and internal cabling
• ¼ mm strip pitch• 200 channels (128
instrumented)• Board mounted series
resistor for back-termination and LP filtering.
• In-vacuum signal cabling using UHV-compatible flex-circuits
• High resolution area can be moved by swapping connectors
9/21/07 IPM Review, September 21, 2007 A. Jansson 8
Microchannel plate
• Max gain with 36 proton bunches is ~1e4 to avoid saturation.
• Can be achieved with single plate
• With dual plates, each plate would run at a very low gain and low bias current.
• Use single MCP with extra-high bias current.
Front end cabling
• 40 channels in each vacuum DB50 feed-thru (flex circuit)
• Transition board from feed-thru.
• Mini-coax bundles with external shield.
• Custom (CMS) DB coax connector in FE board.
9/21/07 IPM Review, September 21, 2007 A. Jansson 9
9/21/07 IPM Review, September 21, 2007 A. Jansson 10
DAQ system
• CMS-QIE chip digitizes signal in tunnel.
• Serial data uplink on optical fiber.
• Receiver and data buffer in upstairs PC
• Timing + QIE clock + QIE clock supplied from PC thru cat-5E cable
Timing card(PCI)
Timingfanout
QIE cards(16x 8 ch)
Data Buffer(2x 8 links)
(PCI)
Host PC (LabView)
Service buildingTunnel
9/21/07 IPM Review, September 21, 2007 A. Jansson 11
QIE card
• 8 channels (CMS QIE8) per board.
• Achieved noise ~1.8fC with 4’ cable.
• Data is combined with timing information, serialized by CERN GOL ASIC (rad hard) and sent thru optical fiber at 1.1Gbps data rate
• Timing fanout board cleans up and distributes clock and timing signals
FE board input/output format
9/21/07 IPM Review, September 21, 2007 A. Jansson 12
7 6 5 4 3 2 1 0
QIE0
15 14 13 12 11 10 9 823 22 21 20 19 18 17 1631 30 29 28 27 26 25 24
QIE1QIE2QIE3
QIE5QIE6QIE7
QIE4
QIE4CAPID0Timing bits
TX_EN/RX_DV
QIE0QIE1QIE2QIE3
QIE5QIE6QIE7
QIE4
QIE4CAPID1Timing bits
QIE0QIE1QIE2QIE3
QIE5QIE6QIE7
QIE4
QIE4CAPID2Timing bits
Proton Revolution CounterError bits
QIE mode
QIE mode
QIE mode
2RF/7 Clock Counter
11000 No effect11001 Proton Injection Marker11010 Antiproton Injection Marker11011 QIE Mode0 (QIE in Calibration Mode)11100 QIE Mode1 (QIE in Mode)11101 QIE Mode2 (Link test - outputs a 32-bit counter to GOL)11110 QIE Mode3 (Reset QIEs)11111 Board Reset
• 15MHz clock frequency• Proton and pbar revolution markers•Encoded signal:
INPUT (Cat5 cable)
OUTPUT (1.6Gbps serial link)
9/21/07 IPM Review, September 21, 2007 A. Jansson 13
Data buffer card
• Handles 8 incoming optical links (64 channels, 1.1 GB/s of data)
• Can sparsify data on–the-fly based on timing masks
• 512MB RAM allows for– 20.000 turns of continuous data– 90.000 turns for 72 bunches– 6 million turns for a single
bunch
• Read out thru PCI64 bus.• Two boards are used to
handle 128 channels.• Sync via jumper cable
• IPM buffer board doubled as prototype for BTeV L1 data buffer.
• Considered for use in MICE experiment.
9/21/07 IPM Review, September 21, 2007 A. Jansson 14
Timing card
• Produces the 15MHz (2/7 RF) QIE clock
• Decodes and transmits beamsync revolution marker + injection and trigger events
• Controls QIE settings.
396 ns (21 buckets)
p-pbar separation ~120ns
RF
2/7
9/21/07 IPM Review, September 21, 2007 A. Jansson 15
Labview Software
Installation timeline
• Magnets installed shutdown ‘04• Vertical detector installed Dec’05
– Arcing problems inhibited operation at full B field. High outgassing from retrofit retainer.
• Vertical detector fixed, horizontal installed May’06– reduced DAQ system with single buffer board
due to power supply limits and firmware problems).
• Full DAQ, larger leak installed Oct07– 80 channels per plane, 2 buffer boards
10/15/2009 IPM status October 2009
9/21/07 IPM Review, September 21, 2007 A. Jansson 17
System Status 2007
• Both IPMs had ‘new’ software and firmware.
• Vertical IPM works pretty well, basicall since installtion.
• Horizontal IPM has some hardware problems (bad FE baord) that required an access.
• Horz IPM also requires two buffer boards to measure beam size at injection
NB. REAL DATA!
9/21/07 IPM Review, September 21, 2007 A. Jansson 18
Beam size at IPM location
By moving detector, able to fit beam in active area, except horizontal beam size at 150GeV
EARLY MEASUREMENTSBrief overview of IPM history
10/15/2009 IPM status October 2009
9/21/07 IPM Review, September 21, 2007 A. Jansson 20
Protons at low beta
• Proton bunch #1 at low beta during store #4758.
• Measured beam size 0.55mm, turn-by-turn variation (noise) 20µm.
• Total signal per bunch ~1.3pC.
30 turn average
single turn
Magnet at 200A
9/21/07 IPM Review, September 21, 2007 A. Jansson 21
Injection – Full B field
0.2 0.4 0.6 0.8
0.1
0.2
0.3
0.4
Proton bunch #21 turn-by-turn
1 cm
Store #4772 Magnet at 200A
RMS profile width
0.9
1.0
1.1
1.2
1.3
[mm
]
0.0 0.5 1.0
0 10 20
FF
T0 10 20 30 40
0
5
10
15
20
25
30
0
35
turn
s
9/21/07 IPM Review, September 21, 2007 A. Jansson 22
100 turns
Four injection measurements superimposed
9/21/07 IPM Review, September 21, 2007 A. Jansson 23
Measured mismatch vectors
• Nominal settings• Q701 at -22A• Q701 at -42A• Q711 at -25A
• Need to increase Q711 by ~15A (saturation?)
• Should result in 5-10% smaller vertical emittance
• Should have very little effect on horizontal plane.
• Should have been tested this morning…
-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
9/21/07 IPM Review, September 21, 2007 A. Jansson 24
Flying wire comparison
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 4 7 10 13 16 19 22 25 28 31 34
Bunch
Bea
m s
ize
(mm
)
Flying wire
IPM
y = 1.0089x
0
0.5
1
0 0.5 1
Flying wire
IPM
Known electronics problem (cross talk from revolution marker pulse)
Comparison of vertical beam size from IPM and nearby Flying Wire.Tuning of abort gap cleaner timing had caused blow-up of certain bunches. From MAD lattice file, expect a 13% wider beam at Flying Wire. See ~1%.
9/21/07 IPM Review, September 21, 2007 A. Jansson 25
Pbars at 980 Gev
Store #5172, Jan 07Horizontal Vertical
9/21/07 IPM Review, September 21, 2007 A. Jansson 26
Pbars at 150 GeV
• Also saw coasting pbars (with vertical system) at injection
• Requires moving the detector to pbar orbit.
• Triggering did not work at first attempt (timing event found to be disabled).
Fri Jan 12 2007, shot #5175
9/21/07 IPM Review, September 21, 2007 A. Jansson 27
Shot 5186 – vertical plane
Two consecutive beam revolutions in Tevatron, just at injection of batch 9.
9/21/07 IPM Review, September 21, 2007 A. Jansson 28
Vertical pbar injection data (batch 9)
• Beam position gives consistent oscillation ~0.2mm.• Need to look closer at beam size data, but…• There is a clear frequency component at twice the
tune!
0 2 4 6 8 10turn
2.4
2.6
2.8
3
3.2
sopmm
2.4
3.2
2.8
0 2 4 6 8 10turn
0.3
0.35
0.4
0.45
0.5
0.55
0.6
ezismm
0.3
0.6
0.4
0.5
Beam size [mm] Beam position [mm]
100 100
ISSUESBrief overview of IPM history
10/15/2009 IPM status October 2009
9/21/07 IPM Review, September 21, 2007 A. Jansson 30
Parasitic signals
• Despite very good shielding, beam related signals still present with MCP gain off.
• Very stable from turn to turn -> reference subtraction possible
• Present even on “broken” channels (where no ionization electrons are seen)
0 50 100 150 200 250 300Turn #
0
25
50
75
100
125
150
175
langiSthgnerts
1x1 store at 980GeV ~center anode strip
Timing markersBeam pulse
Buffer board sync
• With two boards, capacitive background appeared unaligned, and background subtraction didn’t work well– Found and fixed
software data alignment bug
– Verified that buffer board sync check works properly
9/21/07 IPM Review, September 21, 2007 A. Jansson 31
PCI card poll glitching SERDES
• Polling card via PCI during measurement was found to glitch the SERDES reciever.
• “Solved” by using interrupt instead of polling, and reorganising code to minimize PCI traffic during acquisition.
9/21/07 IPM Review, September 21, 2007 A. Jansson 32
INTEGRATIONWhat happened more recently?
10/15/2009 IPM status October 2009
IPM Review 2007
IPM Review Committee ReportThe charge to the committee has consists of the following two items: • 1. What is the current status of the Tevatron IPM systems - the front-end electronics
and DAQ in particular? Identify any known problems or issues.• 2. Can the IPM system be made fully operational - complete detector readout with
production electronics and DAQ issues resolved - by Jan 1, 2008? What additional resources ($$, effort) would be needed to meet that date?
The committee was positively impressed with quality of the work and the part of the presentation devoted to the item 1. Nevertheless very little was said about the second half of the charge. It is unclear:
• What does it mean “to make IPM system fully operational?”• Which additional resources, if any, are required to finish the project timely and
which additional expenses are required?Taking the above into account the Committee suggests to convene an additional short
session devoted to discussion of item 2 of the charge. Andreas should settle with the Tevatron department what they consider for the project to be operational and to present a clear answer to the part 2 of the charge. Having this session in about one or two weeks should leave enough time to elaborate details.
10/15/2009 IPM status October 2009
Tev dept wish list
• In General the IPM controls and data retrieval should be very similar to the flying wire front end. It should be able to perform the following:
• – Decode V:CLDRST state and setup the instrument based on a state to instrument spec mapping owned by the
IPM front end. The TFW instrument has an example of this.– Should provide plane and particle sigma data on event $75 (profile timer) There are 35 profiles that currently
go out up the ramp and through the lowbeta squeeze.– Should provide data to SDA. The instrument should own a “data ready” device similar to TFW that
communicates the state of pending data. This can also be used for lumber jack Client Logging.– The instrument should be free running providing data during a collider store at ~ 1/30 min. (similar to TFW).– The instrument should write bunch by bunch data into ACNET array devices following the standard convention.
• . The ACNET array devices that hold bunch by bunch information should be 37 elements long. • The enumeration of array devices should have the [0] element of the array as the average value of the
36 array members to follow. There may have to be some special code to handle updating the average to reflect and average of elements with non zero values.
• The [1 – 36] would store the data where bunch 1 would correlate to [1] of the ACNET device respectively.•
– ACNET parameter representing data and setup should be placed in the lumberjack utilizing Client Logging.– The IPM should have an ACNET application that can perform the following functions (similar to T46 for TFW):
• Standard user interface.• View and Configuration of State to IPM Spec Transition Map for pre-loading specs.• View and Configuration of IPM specs.• Ability to access and plot profile data correlated to data taken on profile event and periodic sampling with
capability to have fitting options for plots.– IPM ACNET devices should attempt to follow a naming convention.
10/15/2009 IPM status October 2009
Tevatron IPM Specs for Collider II OperationGeneral Collider II IPM Specifications:
ACNET control
• Since early 2008 the Tevatron IPMs are state driven in much the same way as flying wires.
• Selected output is logged when data ready since mid-January 2008
9/3/2008 IPM status A. Jansson 36
Tom’s ACNET interface
9/3/2008 IPM status A. Jansson 37
Available summer 2008 (no plotting)
PSPEC settings
9/3/2008 IPM status A. Jansson 38
Pbar jacking
10/15/2009 IPM status October 2009
Jan 31, 2008Store #3874
Pbar jacking 2
10/15/2009 IPM status October 2009
Feb 1, 2008Store #3876
IPM–FW comparison
9/3/2008 IPM status A. Jansson 41
Jan 2008
How to improve data quality?
• Fluctuations from measurement to measurement much bigger than turn-by-turn fluctuations for coasting beam.
• One known source of bad data: synchronization problems!
10/15/2009 IPM status October 2009
Example: Synchronized
9/3/2008 IPM status A. Jansson 43
Example: Unsynchronized
9/3/2008 IPM status A. Jansson 44
Easy to detect and reject by eye for manual data-taking!
One manifestation of sync issues
9/3/2008 IPM status A. Jansson 45
50 100 150 200 250 3000
50
100
150
200
250
300
P rot on C ounter
Histogram of proton counter in first buffer data frame. Expect single cluster of three bins around zero.Saw wider cluster + scatter + unexpected peaks
Sync saga
• We discovered relatively early on that PCI access during an acquisition would sometimes glitch one or more of the high-speed data links.– Appears to be a FPGA internal problem, now
believe it is caused by does not like 5V input signals on pins close to high speed inputs.
– Work-around: re-order setup sequence to limit access to board, use interrupt instead of polling.
– Seemed to work fine.
9/3/2008 IPM status A. Jansson 46
Board layout
9/3/2008 IPM status A. Jansson 47
Sync saga cont’d
• Even without PCI access during acquisitions, loss of synchronization still occurred relatively frequently, particularly for injection measurements (long wait time)– Improved the initial synchronization logic– Added firmware logic to allow links to
resynchronize dynamically if needed, while waiting for a trigger (In original implementation, the entire chain was synchronized once by a timing card reset just before an acquisition)
9/3/2008 IPM status A. Jansson 48
Sync saga cont’d
• Early resync logic worked in test stand, but not in the Tevatron systems.• Used simulated TVBS in lab5 teststand.
• At this point, FY08 omnibus budget came out, furloughs started and we lost Kwame (our FE board engineer) to Florida.
• Rick (Buffer board engineer) took over debugging of both boards, and we moved the test stand to FCC (closer and real TVBS available).– Real TVBS available
9/3/2008 IPM status A. Jansson 49
Sync saga cont’d
• Found occasionally missing AA markers, and that FE board and buffer board treated this unspecified case differently.– Fixed with buffer firmware modification.
• Also discovered “unexplained” PCI accesses during acquisition?– Don’t know where they come from, or how
long they have been there (system patch?). – Tried to have board ignore PCI requests
during acquisition, but this caused total PC lock-up.
9/3/2008 IPM status A. Jansson 50
Status ~sept ‘08
• Improved re-sync logic got residual error rate down to per mil level (in test stand).
• Couple of “full firmware versions” tested OK in test stand but crashed the horizontal IPM host PC.
• Moved horizontal system to FCC for easier debugging.– Source of problem related to new inter-board
sync logic– Found bad power supply in horizontal system
• Swapped test stand PC with Horz system
9/3/2008 IPM status A. Jansson 51
Summary of sync issues
• Spontaneous front end resets (early on)– Revert to default mode and loss of synchronization.– Due to Single Event Upsets (or just noise on slow control channel?)– Fixed with new front end firmware version plus Labview software changes
• Trigger/Counter synchronization (affects both systems)– Links synchronized, but bunch data not in the correct time-bins.– Due to “missing” AA markers– Fixed by having both boards treat missing markers in same way.
• Intra-board synchronization (affects both systems)– Loss of SERDES lock or illegal character on serial link. – Due to “unexplained” PCI accesses– Effect minimized by re-sync logic. Errors during acquisition flagged.
• Inter-board synchronization (affects mainly horizontal)– Data from two buffer boards not aligned (varying from measurement to
measurement) affecting bunch profiles and background subtraction.– Was hard to debug in the presence of other sync issues…– Fixed with a combination of firmware and software changes
9/3/2008 IPM status A. Jansson 52
LabView software update
• Changed software to not issue “data ready” when a acquisition problem is detected– Only data without hardware errors
datalogged.
• Fixed ”total signal charge” readback, and chi-square readback.
9/3/2008 IPM status A. Jansson 53
Nov’08
Observations with “new” firmware
• The beam related background, which used to be significant but very stable, is now fluctuating a lot in horizontal plane.– Likely due to some sort of timing jitter
• Also, the signals appear to be smaller than before– Possibly due to new x-collimator ion
pump?
9/3/2008 IPM status A. Jansson 54
Beam background
9/3/2008 IPM status A. Jansson 55
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0
2
4
6
8
1 0
1 2
1 4
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0
1
2
3
4
5
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0
2
4
6
8
1 0
1 2
1 4
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0
1
2
3
4
5
“Average beam background”
“Fluctuation in beam background”
VIPM and HIMP both looking at the vertical signal
Various failed attempts to fix jitter
• Swapped timing cards• Changed timing card slot location
(possible interference with buffer board)
9/3/2008 IPM status A. Jansson 56
Status – Jan’09
• Observed jitter in horz clock/timing signals– Not reproducible in test stand– Swapping timing boards back and forth
gave inconclusive results.– Possibly related to fluctuating signal?
• In the meantime found and fixed bug in background subtraction for pbars.– Pbar data much improved
9/3/2008 IPM status A. Jansson 57
Jan’09
February power failure
• After the power outage on February 11, IPMs stopped reporting data to ACNET– Software detected persistent buffer errors bits, marking data as
bad• VIPM: Buffer 1/2 mismatch• HIPM: Pbar marker error
• Swapped cables, to locate error– Buffer mismatch followed upstairs PC– Pbar error followed downstairs crate.
• Pulled buffer boards from vertical system for bench test.– Left horizontal PC monitoring vertical signals
• Bench tested boards, finding nothing out of the ordinary– Upgraded test stand labview code and re-tested, saw nothing
• Reinstalled, buffer mismatch problem gone– Had to distable signals from horizontal front end board 1, which was
causing the pbar marker error.
• Mechanics now seems to be working properly
9/3/2008 IPM status A. Jansson 58
Signal strength issue
• Signals are smaller than they used to be, fluctuating a lot and occasionally disappear altogether– Not due to stochastics of gas
interaction– HV rise time ruled out as a culprit(?)– Gas pressure fluctuating?– Not currently understood
9/3/2008 IPM status A. Jansson 59
April’09
Fluctuating data
9/3/2008 IPM status A. Jansson 60
OK data
9/3/2008 IPM status A. Jansson 61
Marginal data
9/3/2008 IPM status A. Jansson 62
Bad data
9/3/2008 IPM status A. Jansson 63
Before shutdown ’09
• Tried turning off some ion pumps– Signals strength improved somewhat– Signal fluctuations still there
• Took vertical tbt data at injection– No obvious mismatch
• Found almost 1V ripple on 3.3V supply to HIPM timing card– Replaced power supply– Timing jitter now essentially gone
9/3/2008 IPM status A. Jansson 64
Signals just before shutdown
10/15/2009 IPM status October 2009
HIPM at injection
9/3/2008 IPM status A. Jansson 66
Improved error detection
10/15/2009 IPM status October 2009
During shutdown, improved the buffer error Detection to reduce false positives
Error rates in machine (without beam) consistent with test standwith few exceptions (weak boards). Swapped two boards.
STATUSCurrent
10/15/2009 IPM status October 2009
Data availability
• While we have been working on improving the measurement reproducibility, data from ramp, squeeze and store have been available in ACNET and logged since beginning of 2008.
• Injection and “pbar jacking” data taking have been enabled occasionally– Not enabled permanently due to concerns
about depleting MCP if stuck with high voltage on for long times.
10/15/2009 IPM status October 2009
Hardware, firmware and software
• On-the-fly re-sync logic seems to work
• Intra-board sync seems to work• Improved the error detection,
minimizing false positives• Small residual error rate (due to
SERDES glitches) remain. Any flawed data is discarded
10/15/2009 IPM status October 2009
Signal Quality
• Signal is weaker than before, in particular for horizontal system– Horizontal pbar signal no longer visible– Leak no longer as effective?
• Signal level is fluctuating significantly, causing fluctuation in measured beam sigma.– Gas puffs from pumps?
• Noise level also seem somewhat higher, at least in some channels
10/15/2009 IPM status October 2009
E0 gas pressure
10/15/2009 IPM status October 2009
Last 4 years
TSPs were fired during the shutdown
Degassing TSP filaments at IPMs (spoiling the TSP)Helped bring the signal back up somewhat
TSP degassing test
10/15/2009 IPM status October 2009
Trying to raise local pressure by degassing TSP filaments gavelittle or no effect on the measured signal (might have worked better if pumps were off)
Horz signal then and now
10/15/2009 IPM status October 2009
• 2007, before shutdown and now
IPM vs FW
9/3/2008 IPM status
Store #7243Oct 10, 2009
Consistent to 10% with FW, but weak IPM signals
Turn-by-turn data
10/15/2009 IPM status October 2009
Enabled injection measurement for store #7265
Got data!No apparent vertical mismatch (horizontal signal too weak)Some trigger issues (only a few bunches captured, rest of data stale)To be looked into further.
Summary
• Tev IPMs are delivering data since almost two years
• Compares quite well to FW, but some fluctuations in the sigma readings.
• All known hardware issues have been resolved, data affected by any remaining occasional hardware glitches discarded.
• Remaining issues related to weaker signal with larger fluctuations than before.
• Also seems there are some triggering issues for injection measurement (not used regularly)
10/15/2009 IPM status October 2009