IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 1
Liquid Cooling: An Update
Dr. Mikhail Spokoyny,
EEPT Lab, A.J.Drexel Plasma Institute, Philadelphia PA, [email protected]
in cooperation withCool Technology Solutions, Inc. San Diego CA, USA
1April 11, 2012
IEEE Santa Clara Valley CPMT Society Chapter
Santa Clara, CA
ContentContent
1. Main Trends in meeting challenges of thermal management within computer industry and power electronicsLi id C li i h f th li ti2. Liquid Cooling – niche for these applications
3. Current Conditions of the liquid cooling market - main applications, standard and new technologies
4. New Direction in liquid cooling – Submerged Jets5. Different Technologies within Submerged Jets and
main results of their implementations 6. Conclusions
2
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 2
Integration Density of Digital Semiconductors
1. More transistors require more power
2. More power produces more heat to be dissipated
3. Faster microprocessor more frequency - more heat to be dissipated
4 Next step Multi core4. Next step – Multi-core architecture
5. More cores – higher heat flux – more challenges for heat dissipation
3
Heat Density Challenge
160
Power dissipation3.00E+06
Heat Flux
Intel Xeon processor family– Servers, 7000 series
20
40
60
80
100
120
140
Foster MP
Cranford
Paxville
Tulsa
Dunnington‐4
core
Dunnington‐6
core
Beckton
Westm
ere
Dra
ke
5.00E+05
1.00E+06
1.50E+06
2.00E+06
2.50E+06
Drake Foster MP
Cranford
Paxville
Tulsa
Dunnington‐4
core
Dunnington‐6
core
Beckton
Westm
ere
4
00 1 2 3 4 5 6 7 8 9 10
F D
0.00E+000 1 2 3 4 5 6 7 8 9 10
Years of the 21-st century
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 3
*Heat Density Challenge
5* Prof. A. Bar-Cohen, IMAPS, AWT, 2011
*Power Electronics: Evolution in Wind-Energy
Engineering
• Increase of peak power up to 7 MW per unit
• High power IGBT-modules and invertersmodules and inverters are needed for high efficiency energy conversion
• 1% loss of efficiency at 7 MW = 70 kW heat
Consequences:
6
qIncrease of system power
= larger power dissipation
MORE HEAT
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 4
In Power Electronics More HEAT is generated due to:•General increase of power capacity of semiconducting devices•Even the most efficient circuitry design cannot negate negative effects of the absolute increase in power dissipated by a device
In Digital Computational Devices More HEAT is generated due to:•Increase of computational power due to higher clock rates•Higher performance by multi-core design architecture
Solution: •Efficient Thermal design•Implementation of appropriate cooling method
7
p pp p g•Selection of high-efficiency cooling systems and components
Cooling Systems. Classification
Passive Cooling S t
Controlling Cooling S t
Active Cooling S t
• Passive Cooling System – dissipates heat at the level at or above ambient temperature without possibility of Tj control – Constant current and voltage fans, radiators, cold plates.
• Passive Controlling Cooling System –
Active-Controlling
Cooling Systems
Systems Systems
Passive-Controlling
Cooling Systems
SystemsHybrid
Systems
Hybrid Cooling Systems
Passive Controlling Cooling Systemdissipates heat at the level at or above ambient temperature with a possibility of Tj control – Variable speed and voltage control fans, cold plates with variable flow pumps.
• Active Cooling System – dissipates heat at the level below ambient temperature without possibility of Tj control – Constant voltage thermoelectric devices, Stirling micro refrigerator, closed loop chillers.
• Active Controlling Cooling System – dissipates
8
SystemsSystems • Active Controlling Cooling System dissipates heat at the level below ambient temperature with possibility of Tj control – Constant voltage thermoelectric devices, Stirling micro refrigerator, closed loop chillers
• Hybrid Cooling System – dissipates heat at any desired level. At the level at or above ambient temperature – Passive Cooling is employed, while at the level below ambient temperature Active Cooling kicks in.
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 5
Cooling Systems ClassificationMain schemes of energy dissipation
Passive Cooling Systems
Heat
Object Carrier
ObjectRadiator
Passive Cooling System – AIR Heat is dissipated to the ambient air
Object Carrier
Object
Heat pipe
Object Carrier
Object
Cold Plate
Pump
Object CarrierObject
Cold Plate
Pump
Liquid - air heat exchanger
HeatRadiator-in condense
zone of a heat pipe
9
Passive Cooling System Two phase – AIRHeat is dissipated to the ambient air
Passive Cooling System – LIQUID Heat is dissipated to the house water
Passive Cooling System – LIQUIDHeat is dissipated to the ambient air
exchanger
Vapor chamber can be utilized instead of a heat pipe
Object Carrier
Object
Object Carrier
ObjectObject Carrier
Object
Active Cooling Systems
Cooling Systems ClassificationMain schemes of energy dissipation
Pump
Cold Plate
HeatHeat
Radiator
Object
Cold Plate
PumpChiller
TEM*
Fan
TEM*
Active Cooling System – AIRHeat is dissipated to the ambient air
Active Cooling System –LIQUID heat is dissipated to the house water
Active Cooling System LIQUID
10
Controlling Cooling Systems vary fan speed or after the chiller liquid temperature depending on the temperature of the cooled object
to the ambient air Active Cooling System – LIQUID Heat is dissipated to ambient air
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 6
Controlling and Hybrid Cooling Systems
Obj C i Object Carrier
Cooling Systems ClassificationMain schemes of energy dissipation
Heat
Object Carrier
ObjectRadiator
Object Carrier
Object
Cold Plate
Pump
Chiller
Heat
Fan
Cooling Systems Controlling is based
Passive Controlling Cooling System –controls fan’s
Object Carrier
Object
Cold Plate
Pump
TEM*
Cool Technology SolutionsMulti-functional Control
System
Tcase
11
Controlling is based on Tj or Tcase temperature of the object being cooled
controls fan s voltage and speed
Active Controlling Cooling System –controls TEM’s voltage and/or pump’s voltage and speed
Active Controlling Cooling System –controls chiller’s temperature set point
The heat convection is ALWAYS part of heat dissipation because heat from OBJECT may dissipate outside only to air or to Liquid (house water, river, sea)
Type of convection
Heat Transfer Coefficient “h”,
W/m2K
Advantage Disadvantage
Natural air convection
From 5 to 10 Cost: zero, very compact
Dissipated heat is limited to no more than 2 W
Forced air convection
From 10 to 200 Very efficient and practical.
Dissipates heat in applications up to 300 W
Acoustic noise from fansRequires highly developed
dissipating surface
Laminar liquid convection
From 5 to 10^3 Capable of dissipating up to 800
W of heat Utilizes low pressure,
therefore low
Expensive.Requires cold plates with micro
structures to significantly increase contact area
12
Vers. 1.0 -2012 12
acoustic noise pump
Turbulent liquid convection
From 10^3 to 5*10^5
Capable of dissipating up to 30
kW
Requires usage of high pressure powerful pumps
Two phase : water boiling, vapor
chamber..
Two phase water boiling10^6
Can dissipate almost any amount of heat
Next to impossible to create compact solutions. Require
special device for vapor condensation
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 7
Heat Transfer / Convection
Mechanism: •Entrainment of thermal energy in a flowing liquid or gas.Qh = h*F * (Tf -Tavr.env)Qg = G*c*(Tout – Tin)Qh = Qg; S contact s rface area F s rface
Surface F with fins, or …
Outlet: Hot air to ambient or to liquid☟
Hot ObjectQg
S- contact surface area, F- surface area of radiator or cold plate; F/S-coefficient of finned surfaces
Utilization of lower Tavr.env (cryogenic liquids) to increase dissipated heat is impossible in semiconductors industry as many semiconducting devices (example: transistors in microprocessors’ technology with less
Liquid-Air Heat exchanger
Q Qh
Q SF
13
Vers. 1.0 -2012 13
microprocessors technology with less than 60 nm) are freezing at temperatures below minus 30°C Inlet: Cold air or liquid
G- fluid flow ratio
Main scientific and engineering direction of Cool Technology Solutions, Inc.: development of devices where turbulent convection is achieved artificially at low media velocities (at velocities 8 to 10 times lower than where usual laminar-to-turbulent transfer happens)
Main Liquid Cooling Technologies on a MarketMain Liquid Cooling Technologies on a MarketContact surface is comprised of:
- pins, plates, pin-fins with some increasing properties using diamond pins, pimples, dimples and etc- Micro-channels as the main trend
Most modern heat transfer devices implement:-Synthetic Jets - Impingements Jets - the main trend
UL
D B
E N
EX
T?
....
Do
f T
EC
HN
OL
OG
Y S
HO
U
Industry/University Cooperative Research Centers Program - http://www.nsf.gov/pubs/2002/nsf01168/nsf01168ff_photo_02.htmSingle-phase, miniaturized convective cooling- http://www.zurich.ibm.com/st/cooling/convective.html
Highest results of hydraulic performancesImpingements Jets
Highest results of thermal performancesMicro-channels
WH
AT
KIN
D
14
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 8
SUBMERGED JETSSUBMERGED JETSNew Generation of Cooling TechnologiesNew Generation of Cooling Technologies
CONCEPT AND BRIEF SUMMARY CONCEPT AND BRIEF SUMMARY The first implementation of Submerged Jets Cooling Technologies family, new single-phase liquid cooling system* - Collider JetTM Cooling Technology - was unveiled inSeptember of 2010 at Intel Development Forum.This cooling system for processors had utilized sets of jets directed towards each otherwith relative micro shift on the central axes snuffed to achieve sharp artificialt b li ti f t Alth h t th b t i th “f il ” it i t ti fU
LD
BE
NE
XT
?...
.
turbulization of streams. Although not the best in the “family”, it is representative ofphysical processes that take place inside, and therefore deserves detailed description.
Do
f T
EC
HN
OL
OG
Y S
HO
U
*Volodymyr Zrodnikov, Mikhail Spokoyny, “Interlocked jets cooling method and apparatus”, Patent Appl No.: US 2011/0042041 A1, Date of Patent: Aug.18, 2010.
WH
AT
KIN
D
Concept of Jet Cooling Technology
15
Heat sinks were made of copper and aluminum and had the following basic geometrical characteristics:
JET COOLING SYSTEM FOR MICROPROCESSORSDESIGN REALIZATION, CFD and Experimental Data
Overall dimensions Array of Jets dimensions
UL
D B
E N
EX
T?
....
20 x 26 x 14 mm
30 x 46 x 26 mm
55.6 x 55.6 x 23 mm
88 x 93 x 26 mm
140 x 180 x 26 mm
from 0.05 x 0.05 mm
to 0.9 x 0.9 mm
and Diameter
from 0.1 mm
to 0.9 mm
Do
f T
EC
HN
OL
OG
Y S
HO
U
Pin dimensions
from 0.5 x 0.5 x 3.8 mm to 1.2 x 1.1 x 6.8 mm
Gap between pins
from 0.25mm
to 1.2 mmWH
AT
KIN
D
16
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 9
DIRECT NUMERICAL SIMULATION OF OPPOSING JETS FLOW STRUCTURE
Navier-Stokes equations along with a system of non-stationary, three-dimensional continuity expressions constituted system’s mathematicalmodel. These were resolved using icoFoam and turbFoam solversfrom the hydrodynamic modeling system OpenFOAM along with preand post-processor data preparation systems Salom and Paraview.
UL
D B
E N
EX
T?
....
p p p p y
Direct numerical simulation Computational domain is presented onpictures. As seen from these images, the computational domainrepresents the flow volume of filling the manifold.
Do
f T
EC
HN
OL
OG
Y S
HO
U
Computational domain. General view.
WH
AT
KIN
D
17
UL
D B
E N
EX
T?
....
Computational domain. Bottom view. Computational domain. Frame.
Time-dependent hydrodynamic interaction of opposing jets and coolantflow animation were obtained from simulations.
Do
f T
EC
HN
OL
OG
Y S
HO
U
As an example, on these pictures show typical images of the velocitydistribution U and flow structure in the characteristic section of thecomputational domain during developed steady flow.
The presented results allow to understand the nature of coolant flowinside the heat sink, making it possible to tailor geometrical parametersfor prototyping and fabrication of experimental samples.
WH
AT
KIN
D
18
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 10
UL
D B
E N
EX
T?
....
Typical distribution of flow velocity U in the gap between the pins located in the plane of symmetry of computational domain. Main nozzles 0.7 x 0.7 mm, support nozzles 0.12 x
0.12 mm.
Do
f T
EC
HN
OL
OG
Y S
HO
U
Typical flow structure in the gap between the pins located in the plane of symmetry of computational domain. Main nozzles 0.7 x 0.7 mm, support nozzles 0.12 x 0.12 mm.
WH
AT
KIN
D
19
EXPERIMENTAL RESULTS
The section presents experimental results of the thermal resistance as afunction of the flow rate in aforementioned cold plates. Experimentalsetup’s schematic drawing is shown on the picture. The temperature of theProcessor Imitator was measured by a thermocouple, embedded in thecenter (and ¼, and ¾ of size) of Imitator and connected to a dataacquisition system Coolant’s flow rate varied between 0 1 GPM and 4U
LD
BE
NE
XT
?...
.
acquisition system. Coolant s flow rate varied between 0.1 GPM and 4GPM. Corresponding pressure drops varied between 2 psi and 35 psi.Footprint cold plate 3”x 3” (77mm x 77mm)
Chiller
Flow Meter
C ld Pl t
Tj- temperature junction control
Do
f T
EC
HN
OL
OG
Y S
HO
U
Pressure Drop measurement
Experimental setup’s schematic drawing
Cold Plate
Processor Imitator
jinside of imitator
WH
AT
KIN
D
20
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 11
Comparison results between straight and alternate jets with the same total cross section area
j tem
p.
UL
D B
E N
EX
T?
....
tan
ce (
deg
C/W
) b
ased
on
Tj
Do
f T
EC
HN
OL
OG
Y S
HO
U
Cold Plate’s thermal resistance as a function of the method and number of nozzles
Th
erm
al r
esis
t
Number of nozzles, pc
WH
AT
KIN
D
21
Comparison results between alternate jets and alternate jets with gap support nozzles
j tem
p.
UL
D B
E N
EX
T?
....
tan
ce (
deg
C/W
) b
ased
on
Tj
Do
f T
EC
HN
OL
OG
Y S
HO
U
Th
erm
al r
esis
t
Number of nozzles, pc
Cold Plate’s thermal resistance as a function of the method and number of nozzles
WH
AT
KIN
D
22
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 12
CONCLUSION - 1
Thermal Performances (based on Thermal resistance)
Hydraulic Performances(based on pressure drop)
Best Best
Micro channels
Collider JetsTM –
Alternate horizontal jets
Impingement Jets
Straight horizontal jets
UL
D B
E N
EX
T?
....
Worse Worse
Straight horizontal jets
Alternate horizontal jets
Alternate horizontal jets with gap support jets
Impingement Jets
Micro channels
Collider JetsTM –
Alternate horizontal jets with gap support jets
Alternate horizontal jets
Collider JetsTM Cooling Technology main advantage –
Do
f T
EC
HN
OL
OG
Y S
HO
U
Collider Jets Cooling Technology main advantage –Highly efficient thermal performance (although not as high as for Micro channels)
Collider JetsTM Cooling Technology main disadvantages –High pressure drop (poor hydraulic performances)Inability to dissipate heat from local hot spots
SOLUTION? Next generation of SUBMERGED JETS Cooling Technologies:Vortex AlternateTM, Swirling Jets-Stream TM, and Wave JetsTM
WH
AT
KIN
D
23
Next Generation of SUBMERGED JETS Cooling Technologies –Vortex AlternateTM, Swirling Jets-Stream TM, and Wave JetsTM
Comparison of preliminary results
tem
p.
Thermal Performances vs. Flow Rate
0.55
0.6
CTS-V series- Vortex-Alternate jets
HN
OL
OG
Y
ance
(d
eg C
/W)
bas
ed o
n T
j
0 25
0.3
0.35
0.4
0.45
0.5 CTS-S series -Swirling Jet-Stream
CTS- W series- Wave Jet
he
NE
XT
CO
OL
ING
TE
CH
Flow Rate, LpM
Th
erm
al r
esis
ta
0.1
0.15
0.2
0.25
0.0 5.0 10.0 15.0 20.0
Th
24
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 13
Pressure Drop (bar) vs. Flow Rate
1.4
1.6
CTS-V series- Vortex-Alternate
Next Generation of SUBMERGED JETS Cooling Technologies –Vortex AlternateTM, Swirling Jets-Stream TM, and Wave JetsTM
Comparison of preliminary results
HN
OL
OG
Y
ress
ure
Dro
p, b
ar.
0.6
0.8
1
1.2
CTS V series Vortex Alternate jets
CTS-S series -Swirling Jet-Stream
CTS- W series- Wave Jet
he
NE
XT
CO
OL
ING
TE
CH
Flow Rate, LpM
Pr
0
0.2
0.4
0.0 5.0 10.0 15.0 20.0
Th
25
Preliminary CONCLUSION - 2
Thermal Performances (based on Thermal resistance)
Hydraulic Performances(based on pressure drop)
Best
Impingement Jetslow Re numbers
Best
Micro channelslow Re numbers
Swirling Jets-StreamTM
Wave JetsTM
HN
OL
OG
Y
Collider JetsTM –
Alternate horizontal jets with gap support jets
Straight horizontal jets
Alternate horizontal jetsAlternate horizontal jets
Impingement Jets
Collider JetsTM -
Alternate horizontal jets with gap support jets
Vortex-Alternate JetsTM
Wave JetsTM
Vortex-Alternate JetsTM
Swirling Jets-StreamTM
he
NE
XT
CO
OL
ING
TE
CH
Worse Micro channels
j
New Parameter – Technology’s Ability to dissipate heat from a local hot spot
WorseStraight horizontal jets
Alternate horizontal jets
Th
26
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 14
Ability to dissipate heat from local hot spot(max Thermal Performance for max Heat Flux based on Local Thermal Resistance)
Best
Swirling Jets-StreamTM
Samples of manufactured cold plates and heat exchangers implementing Submerged Jets Cooling Technologies developed by Cool Technology Solutions, Inc.
HN
OL
OG
Y
Preliminary CONCLUSION - 3
Alternate horizontal jets
Impingement Jets
Collider JetsTM -Alternate
Micro channels
Vortex-Alternate JetsTM
Wave JetsTM
he
NE
XT
CO
OL
ING
TE
CH
Worse
Straight horizontal jets
horizontal jets with gap support jets
Th
27
So far, among all existing on the market thermal management technologies ideal can not be found, each one has its disadvantages and limitations.
Usually better thermal performance (efficiency of heat transfer) is accompanied by either worse hydraulic properties, or limited, if any, capability of handling of (to dissipate heat from) a local hot spot.
UB
ME
RG
ED
JE
TS
Preliminary CONCLUSION - 4
New SUBMERGED JETS Cooling Technologies – Vortex AlternateTM, Swirling Jets-Stream TM, and Wave JetsTM (US Patent Pending Applications)are not ideal either, but have already shown unique capabilities and extremely impressive performance.
We are very optimistic about New SUBMERGED JETS Cooling Technologies and it capabilities and prospects, and are pretty sure that for each specific request for any specific application we will be able to find the best applicable technology and solution capable of meeting requested set of parameters.
LIN
G T
EC
HN
OL
OG
Y -
SU
Th
e N
EX
T C
OO
L
28
IEEE Santa Clara Valley Chapter, Components, Packaging and Manufacturing Technology Society
April 11, 2012
www.cpmt.org/scv/ 15
ACKNOWLEDGEMENTS
Author would like to thank Dr. F. Rottman and J. Hernsdorf for their help in preparation of this presentation; Dr. V. Trofimov for performing all simulations; Prof. S. Isaev for his advice and input through our discussions, and President of Cool Technology S l O b f hSolutions, Inc. Mr. N. Ortenberg for managing the entire project.
THANK YOU!ANY QUESTIONS ?
29