ISL Cooling Leak: ISL Cooling Leak: Cause and MitigationCause and Mitigation

Jose E. Garcia FNAL for ISL Task Force

All

Ex

pe

rim

en

ts M

ee

tin

g

2

ISL Repair Task ForceISL Repair Task Force

Jose E. Garcia FNAL

Del Allspach, Mary Convery*, Jose Enrique Garcia, Doug Glenzinski , Ignacio Redondo Fernandez*, Ulrich Husemann, C.M. Lei, Mike Lindgren, Aseet Mukherjee*,

Bill Noe, Robert Roser*, Ken Schultz, David Stuart*, Bob Wagner, Peter Wilson

*not shown in the photo

ISL/L00 Portcards

3

CDF Silicon SystemCDF Silicon System

Jose E. Garcia FNAL

4

ISL CoolingISL Cooling

• Cool sensors, readout electronics on detector (SVX chips) and nearby readout electronics (Portcards)

• SVX and ISL have independent cooling systems:– SVX (-10oC)

• Need to keep SVX colder to reduce impact of radiation damage• SVX 70/30 water/glycol

– ISL (+6oC)• Includes portcards for ISL/L00 on one cooling ring and SVX on a

separate cooling ring• ISL initially used distilled water, added 10% glycol in 2005 to avoid rare

freeze condition

• Each system broken into ~10 cooling circuits on face of central detector with separate electronic control valves

• Operated sub-atmospheric: leak sucks air into system rather than pushing coolant onto detector or electronics

Jose E. Garcia FNAL

5

Problem StartsProblem Starts

• Since March 2006: the conductivity of the ISL system went out of scale.

– Several attempts to bring it down were not successful

– Problem was investigated

• Beginning of 2007: ISL Cooling System degrades– During last month of 2006 the ISL cooling system became more

touchy. After a trip or a power outage the flows used to operate the detector were not easily recovered.

– At the beginning of 2007 some of the electronic valves started to fail. This problem got worse with time.

Jose E. Garcia FNAL

6

Co

nd

uct

ivity

3300

1200 700

8163 0.5

AU

G-0

6

MA

R-0

6

AP

R-0

7

MA

Y-0

7

History ShortHistory Short

Dra

in

DI

Res

in

No

w

• March 2006 ISL conductivity went out of range of the inline conductivity meter– Dashed line shows the estimated behavior of the conductivity

– Numbers show measurements performed by external labs

• Problems were observed on e-valves in February – March 2007.

• pH measured on March was found to be ~ 2 (Vinegar).

Problem was attacked!!!

Jose E. Garcia FNAL

7

System DrainSystem Drain

• In March 2007, pH of the ISL cooling system was 1.9 and the conductivity 3300 S/cm.

• Treating the coolant with chemicals was discarded without a complete risk evaluation.

• Diluting and/or draining the system was the option chosen to obtain the desired coolant quality. This option will also remove the glycol from the system.

• 2 shifts access was requested to perform the draining of the around 700 liters contained in the system.

• The piping system was divided in several circuits and each of them was drained.

• Around 3 – 4 barrels of coolant were taken out of the system. Glycol was almost completely removed (less than 1% concentration remained in the system)

Conductivity = 700 S/cm pH = 2.8

Jose E. Garcia FNAL

Drainings and pH obtained depending on the total of the

volume drained

8

System Drain IISystem Drain II

• The first draining was successful but the pH achieved was still too low. The desired range for the pH was 4 - 6

• Being pH a logarithmic scale leaving even a small amount of coolant (5 to 10%) decreases much the effect of the draining

• Final pH obtained was consistent with a draining of around 80% of the total system coolant (some of the legs in the pipes are difficult to drain).

• To obtain a pH around 4, two or three drains will be needed.

• The conductivity was also decreased by the dilution, but the coolant acidity questioned the effectiveness of the resin.

• Tests in lab showed that at pH ~ 2, the resin worked perfectly. But samples not only had a decrease in conductivity in addition pH increased.

Jose E. Garcia FNAL

9

Conductivity vs AcidityConductivity vs Acidity

• We used a setup with 1/100 of coolant in the system to emulate the effect of a resin bed exchange on the ISL system.

• We tested the effect of several resin exchanges in the conductivity/pH in the system. Obtained a prediction of the quantity of resin needed to get the desired pH and Conductivity.

• We used a larger resin bed to decrease Conductivity and increase pH. Values obtained were:

Conductivity = 8 S/cm pH = 4.5

Our Lab Tests proved that Conductivy and pH are completely correlated.

pH

Co

nd

uct

ivy

(S

/cm

)

Conductivity pH

Jose E. Garcia FNAL

10

AcidityAcidity

WE BROUGTH pH UP, BUT….

… WHY pH DECREASED IN FIRST PLACE???

Jose E. Garcia FNAL

11

Source of AciditySource of Acidity

Formic Acid

Acetic/Glycolic

• Ion chromatography analysis at Argonne National Laboratory showed carboxylic acids, mostly formic acid, as source of the low pH.

• Very likely that these acids came from the oxidation of glycol

• Al becomes vulnerable of corrosion for pH below 2 as Alumina becomes soluble.

• Corrosion resistance is alloy-dependent:– Heat affected zone around junctions in PortCard

manifold most sensitive (alloy: 6061-Al).

Less corrosion resistant junctions

PortCard Ring

ManifoldJose E. Garcia FNAL

• Coolant was successfully neutralized by draining (to 700 Si/cm, pH 2.8) and de-ionizing with resin (to 8 Si/cm, pH 4.5).

• Degradation on West portcard flows stopped after pH was increased.

• No additional flow problems with ladder lines.

• But…

Welds of the aluminium rings which cool the optical transmitters

(portcards) had already corroded.

• Since the system is sub-atmospheric, leaks result in lower flows. In May flow was too low to operate the portcards.

12

Leaks in the ISL CoolingLeaks in the ISL Cooling

Since May and up to the shutdown in August we were able to operate East ISL/L00 (affected 300 pb-1)

Jose E. Garcia FNAL

13

ShutdownShutdown

SHUTDOWN COMES…

…REPAIR IS PERFORMED…

Jose E. Garcia FNAL

Jose E. Garcia FNAL

Cooling Pipes

Manifold

Leaks

14

ISL LeakISL Leak

Wire Chamber

COT

~28 inches

CO

T F

AC

ECOT FACECOT FACE

15

ISL Cooling RepairISL Cooling Repair

• Action was taken during 2007 shutdown. Important point during the operation was to keep the silicon cold and dry (annealing):

– Plastic barrier encloses wire chamber bore, 600 ft3 volume.

– Desiccant based air drier provided 300 ft3/min to the volume. Dew point always below –10 C.

• Cover holes with epoxy from the inside of the pipe using borescopes and catheters

• Repairs performed over a month with 4 people shifts lead by FNAL specialist Ken Schultz.

• Vacuum tests show that the Al lines are as tight as the stainless steel ones (SVX system)

Jose E. Garcia FNAL

16

Repair…Repair…

Hole

Jose E. Garcia FNAL

17

Ladder Cooling LinesLadder Cooling Lines

• Ladder cooling lines were

investigated during the Bore access.

Several lines were opened.

– No corrosion was observed in

the pipes

– The epoxy used in the ladder

cooling lines did not show any

sings of degradation

• Vacuum in cooling lines is worse than

on 2006 but no clear signs on the

lines.

epoxy

Cooling lines

Jose E. Garcia FNAL

18

Corrosion StudiesCorrosion Studies

• Tests were performed to understand the corrosion process occurred in the system and to predict possible future problems.

• All test samples were immersed in acidic solutions at room temperature, visual inspection, mass loss, dimensional changes if any, were checked.

• 4 kinds of solution were prepared using 7 kinds of Al samples and samples of the epoxy used for the repair (DP190).

• pH of the samples was either 2 or 4, adjusted with formic acid.

• No significant mass loss was observed in the samples of the Aluminum immersed in a 4.5 solution • Some loss was measured for some of the Al types in the acidic solution.

More tests are ongoing

Jose E. Garcia FNAL

19

Instrumentation ImprovementInstrumentation Improvement

• During the shutdown improvements in monitoring and instrumentation have been implemented in the system:

– Larger range conductivity probes (0 – 100 S/cm)– Flow switch in the DI circuit

• Better monitoring of the pH in the system:– Weekly samples taken: pH measured and recorded– Periods of high risk sampling increased to daily.

• Improvements and monitoring will continue.

NEW COND. METERNEW FLOW METER

pH

pH has been kept during the shutdown well within

the safe range 4 – 6

Jose E. Garcia FNAL

20

…AND THE SILICON…

Jose E. Garcia FNAL

21

Silicon OperationSilicon Operation

BEFORE SHUTDOWN

Jose E. Garcia FNAL

BACK TO NORMAL

22

SummarySummary

• ISL Task force will continue:– Corrosion studies will proceed analyzing the future behavior

of the system

– Instrumentation has been improved

– Monitoring the pH will be done weekly

– No further degradation observed in the lines

• ISL/L00 East have been recovered for data taking

– The FULL system has been included on several runs/stores

Jose E. Garcia FNAL

23

…END…

Jose E. Garcia FNAL

24

Aluminum CorrosionAluminum Corrosion

Corrosion by acidic coolant

• Analysis by Argonne: 12.2g/L formic acid + 2.5g/L of either acetic of glycolic acid. Consistent with breakdown of gylcol (10% to 9%).

• Formic acid corrosive to Al

• At high concentrations (pH<~4) expect uniform loss

• At low concentration (pH>4) expect pitting corrosion

• In general, heat affected zones are more susceptible to corrosion

Jose E. Garcia FNAL

25

Leak GeometryLeak Geometry

Cooling pipes

Corrosion/leaks observed in Heat affected Zone of the welds

• Manifold - AL 6061-T6, 0.7 mm wall, 127 mm long tube 6 mm ID

• Cooling Rings - AL 5052, 5 mm wall, oval tubes (21 sided)

• Welded - either 5356 or 4043 filler

• Inner surface area: 0.12m2

• Fed by Cilran (Silicon) tubing from COT face

• Impossible to access from outside, isolated by volume full of cables,~60cm in z.

Jose E. Garcia FNAL