LHC Cryogenics Schneider PLCs: Radiation tests at H4IRRAD Mitigation measures

LHC Cryogenics Schneider PLCs:

Radiation tests at H4IRRAD Mitigation measures

M. Calviani, M. Brugger, G. Spiezia (EN/STI) E. Blanco, J.M. Beckers, D. Willeman, P. Durand, Ph. Gayet (EN/ICE)

2 M. Calviani, E. Blanco - EN Department

Outline

Schneider PLCs issues on the LHC CRYO Control system

H4IRRAD test campaign results Proposed mitigation measures

LMC, 9/Nov/2011


Jan-

11

Feb-

11

Mar

-11

Apr-1

1

May

-11

Jun-

11

Jul-1

1

Aug-1

1

Sep-

11

Oct-1

1

Nov-1

1

Dec-1

1

Jan-

120

1

Apr-

11

May-

11

Jul-11

Jun-1

1

Jul-11

Sep-1

1

Oct

-11

Cryo Schneider PLC: SEU candidates 2011

UX85_QURCB (P8) US85_QURA (P8) US45_QURA (P4) UX45_QUI (P4) UX65_QURCBCC (P6)

LHC downtime due to the Cryo Schneider PLC

issues Is radiation the cause?

Potential SEUs: 7

LMC, 9/Nov/2011

All underground!

PLC Schneider Premium LHC Cryo Surface Cavern

P18 6 3

P2 6 4

P4 11 4

P6 11 6

P8 10 4

Total 44 21 65


LHC refrigerators control

architecture

RMSLSector L (3.3 Km)

4

LHCA

QURA

LHCCA

QURCA

QSCCA

LHCCB

QSCCB

LHCB

QSRB

QSCB

QUI

QSDNQSAA

Comp 4.5K Comp 1.8KMain DryerComp 1.8K Comp 4.5K

QURCB

Cold Box 4.5K

LN2 Buffer

CB 1.8KCB 1.8K

Connection Box

UCB 4.5K

QSRA

QSKA

QSCA

QSAB

Main Dryer

Local & CentralControl Rooms

SCADA Data Servers

RMS

R

Alcoves

Sector R (3.3 Km) Tunnel

Cavern

Surface

Shaft

QSDNRM

PAProfibus DP

WorldFIP

Return Module SL & SR

UX85: ✓✓✓

UX45: ✓

US65: ✓

US45: ✓US85: ✓

LMC, 9/Nov/2011


Radiation tests Objectives

Confirm PLC radiation sensitivity: as shown in the LHC cryogenics control system during 2011 operation

Not for testing PLC radiation hardness Validate potential solutions

Architectures to test:

1. PLC Premium CRYO-like LHC Cryogenics like sample but in two different

configurations: P575634M with memory card P576634M without memory card

2. PLC Premium with remote I/O backplane PLC CPU in safe area (UL) and a passive bus X

connection to the remote I/O backplane located in radiation area.

3. PLC Quantum Redundant architecture Possible solution to increase availability (should be

seen decoupled of the radiation issue)

LMC, 9/Nov/2011

1

2

3


H4IRRAD

SPS beam

Production target

Internal irrad zone (LHC tunnel)

External irrad zone (LHC shielded zones)

Access is possible only from the top shielding (removal of 160 cm Fe blocks)

LMC, 9/Nov/2011

H4IRRAD can reach 1 year of nominal LHC in ~1 week (underground zones)

LHC RadMon for monitoring – agreement with FLUKA simulations within 20-30%


Radiation sensitivity

The PLCs have been tested during a period of low beam intensity due to their significant radiation sensitivity Crashes observed after ~5-10 minutes at H4IRRAD nominal

beam (~5*105 HEH/cm2/min) Clear correlation between PLCs CPU crash and radiation Cross-section estimated from a period with ~5*104

HEH/cm2/min (US85 ~ 3.5*102 HEH/cm2/min)

~5*104 HEH/cm2/min

~2*105 HEH/cm2/min

~4.5*105 HEH/cm2/min

LMC, 9/Nov/2011


Sensitivity resultsType HEH to failure Status

PLC Premium (low memory user application occupancy: ~ 7% exec code )

5*106 HEH/cm2/failure

Not responding + restarted

PLC Premium (P8 QURCB-like application: ~ 18% exec code)

2*106 HEH/cm2/failure

Not responding + restarted

Remote I/O backplane (cabled I/Os)

≥5*109 HEH/cm2/failure

OK!

PLC Quantum redundant

3*106 HEH/cm2/failure(increased availability)

PLC malfunction PLC Premium cross-section in agreement with LHC operation data

Radiation sensitivity appears to be dependent to the user application memory occupancy (~ 2.5 times)

Crash with no remote access possible with PLC Premium! No failure observed or induced by the remote I/O backplane

LMC, 9/Nov/2011

1

2

3


LHC radiation levels in P4/P6/P8

P8: LHCb luminosity Values ≤4*107 HEH/cm2/2011 (in US85) Evolution will directly depend on the cumulated LHCb lumi (1-2

fb-1 2011/2) – same value expected for 2012

P4/P6: beam-gas due to pressure spikes in the cavities (P4) and in the extraction septa (P6) Values ~106 HEH/cm2/2011 in UX 2012 levels should be similar as this year After LS1 values will depend on the vacuum behaviour with 25ns

operation (situation not yet clarified)

P2: ALICE luminosity levels insignificant in US25

LMC, 9/Nov/2011


Conclusions of the test campaign

What we confirmed: High sensitivity to radiation (SEUs) of Schneider Premium

PLCs SEU induced failure cross-section in agreement with LHC

operation data during 2011

What we observed: Radiation sensitivity dependent on the user application

memory occupancy (to be confirmed with Schneider) Radiation robustness of the remote I/O backplane and I/O

cards (including CPU reliability with that architecture) – no failure observed

Equal sensitivity with or without flash memory card Redundancy solution offers limited radiation sensitivity

robustnessLMC, 9/Nov/2011


Mitigation actions(winter shutdown)

1. Improve availability : The radiation campaign in H4IRRAD showed remote I/O low radiation sensitivity and, therefore, the PLC CPU relocation to a safe area (UL) is the solution retained. Feasibility discussed already within the R2E activities (EN/ICE, EN/EL, TE/CRG) Action prepared for the winter shutdown (including ordering, cabling, electricity) Affecting P4, P6 and P8 (in that precise order). P2 not needed

2. Other improvements (once the PLCs are in a safe area)1. Remote accessibility

- Additional Ethernet card to be installed in the relocated PLCs. (21) - Allows remote access via IP reducing time and resources during interventions

2. Robustness - Eliminate flash memory cards in some PLCs (9, awaiting Schneider validation) - Already done in 2011 for the P8 QURA PLC

3. Diagnostics - Enrich the onboard PLC diagnostics (collaboration with Schneider Electric)

LMC, 9/Nov/2011

1

2

3


Mitigation actions(Long Term)

Long-term actions (LS1) Awaiting 2012 LHC operation to decide the next move

(reliability of Schneider PLCs) But in parallel:

1. Continue radiation testing on H4IRRAD facility on equipments like the ones remaining in radiation areas (PLC Quantum, Siemens on turbines,…) and also in other architectures (redundancy)

2. Study a possible full relocation of all active electronics in P4 and P6 as done in P8

3. Validate PLC redundant architectures for a possible migration of some equipments during LS1This is independent of radiation issues and will increase the availability of the cryogenics control system in view of the large downtime induced by an issue in the control system (only if the new situation does not satisfy the required availability!)

LMC, 9/Nov/2011


Acknowledgments

Many thanks to: EN/MEF for the precious support during construction and

during each installation phase The H4IRRAD team and EN/STI for the follow-up of the

test area operation (radiation and beam monitoring) EN/ICE for availability and reactivity

Thanks a lot for your attention

LMC, 9/Nov/2011

14 M. Calviani, E. Blanco - EN Department LMC, 9/Nov/2011


H4IRRAD external zone

p+ beam

EN/EL

EN/ICE PLCs

LMC, 9/Nov/2011


H4IRRAD configuration

SPS beam

Production target

Internal irrad zone (LHC tunnel)

External irrad zone (LHC shielded zones)

Access is possible only from the top (removal of 160 cm Fe blocks)

LMC, 9/Nov/2011


Passive vs. active bus X solutions

The active bus X includes two electronic pieces in both ends of the cable that can be a source of problems under radiation environment.

LMC, 9/Nov/2011


H4IRRAD test area overview

What is it? Test area to test equipment in LHC-like particle

spectra H4IRRAD can reach 1 year of nominal LHC in ~1

week (tunnel and alcoves) Radiation monitoring

Radiation monitoring based on the LHC RadMon system

FLUKA simulations employed to cross-check and understand the radiation field (agreement within 20-30%)

LMC, 9/Nov/2011

Date post:	02-Jan-2016
Category:	Documents
Upload:	xerxes-rogers
View:	18 times
Download:	0 times

LHC Cryogenics Schneider PLCs: Radiation tests at H4IRRAD Mitigation measures

Documents