Future of Cooling High Density Equipment€¦ · Cooling Presents an Opportunity For Energy Savings...

Future of Cooling High Density Future of Cooling High Density Equipment Equipment

Steve MadaraSteve MadaraVice President and General Manager Vice President and General Manager Liebert Precision Cooling BusinessLiebert Precision Cooling BusinessEmerson Network Power Emerson Network Power

2

2007 IBM Power and Cooling Technology Symposium © Copyright 2007 Liebert Corporation. All rights reserved.

AgendaAgendaAgendaIssues facing the Data Center– Managing the explosive growth– Power consumption getting the attention of Governments

Developing the right cooling strategy for today and into the futureExamples of how the right cooling strategy can lower the Total Cost of Ownership

Myth – High Density computing is more costly

3


Managing High Density Servers Managing High Density Servers Managing High Density Servers

200028 x 2U Servers2kW Heat Load

200242 x 1U Servers6kWw Heat Load

20066 Blade Centers24kW Heat Load

2009Rabid Blades

40kW Heat Load

Heading Toward 50kW

200028 x 2U Servers2kW Heat Load

200242 x 1U Servers6kWw Heat Load

20066 Blade Centers24kW Heat Load

2009Rabid Blades

40kW Heat Load

Heading Toward 50kW

Rack density trend when fully populated with the

newest server technology

How most sites are dealing with server density

4


Progression to High DensityProgression to High DensityProgression to High Density

1 kW

2 kW

5 kW

10 kW

15 kW

20 kW

Rack kW

Issues facing the IT ManagerGetting Air out of the RacksHot air mixing with the inlet of other racksDiversity of loads in the Data CenterNot aware that more Fans create heatFlexibility of “On Demand Cooling”

The average server replacement cycle is 3-4 years

5


DCUG Survey Results – What are the Biggest Issues facing the IT ManagerDCUG Survey Results DCUG Survey Results –– What are the What are the Biggest Issues facing the IT ManagerBiggest Issues facing the IT Manager

22%

19%18%

8%

7%7%

5%

4%3%

2%2%

1%

1%0%

0% 10% 20% 30% 40% 50%

Heat Density (Cooling)Space Constraints/Grow th

Pow er DensityAdequate Monitoring Capabilities

Availability (Uptime)Technology Changes / Change

Energy Costs / Equipment Eff iciencyOther

Security (Physical or Virtual)Data Center Consolidations

Data StorageHardw are Reliability

Regulatory ComplianceStaff ing/Training Limitations

6


Air Movement

12%

Electricity Transformer/

UPS10%

Lighting, etc.3%

Cooling25%

IT Equipment50%

Source: EYP Mission Critical Facilities Inc., New York

Cooling Presents an OpportunityFor Energy SavingsCooling Presents an OpportunityCooling Presents an OpportunityFor Energy SavingsFor Energy Savings

Data Center Power Draws

CoolingAbout 37%-

45%

Sources of Energy WasteFans / Blowers running on redundant unitsLack of air containment (cable openings, room leakage)Unnecessary cooling unit cycling on and offLack of humidification control between unitsMixing of Hot and Cold air lowering the effectiveness of the cooling unitExcess Fan energy that turns into heat

7


US EPA Report Data Center Efficiency Public Law 109-431US EPA Report Data Center Efficiency US EPA Report Data Center Efficiency Public Law 109Public Law 109--431431

Energy consumed by servers & data centers has doubled in last 6 years and is expected to double again in next 5 years to > 100 Billion kWHState-of-the-Art technologies and best management practices could reduce electricity use by up to 55%Recommendations include:– Standardized performance measurements for data centers and

equipment– Incentive programs– Research and development– Industry collaboration and partnerships

www.energystar.gov/ia/partners/prod_development/downloads/EPA_Datacenter_Report_Congress_Final1.pdf

8


Understanding The BasicsUnderstanding The BasicsUnderstanding The BasicsHeat generated is directly related to the server power (100%)As server power increases (kW), the airflow (CFM) through the rack increases proportionallyRaised floor tiles are limited in airflow (about 500-1000 CFM)Higher entering air temperatures on a cooling unit will provide more capacity and increased efficiencyHigher density servers will have a greater range of temperatures leaving the rack over time (larger swings of server load)Fan horsepower to move the air is significant and all the power turns into heat (100 kw Cooling unit uses typically a 10 HP motor that generates 8.5kW of heat)

9


Having The Right High Density Cooling Strategy Delivers. . .Having The Right High Density Cooling Having The Right High Density Cooling Strategy Delivers. . .Strategy Delivers. . .

Cooling for High Density racks– Increase servers per rack– Increase number of racks per room

Energy Efficiency– Lower Operating Costs– More power allocated for IT/Server loads

Get more out of your existing facility

10


0 5 10 15 20 25 30 35 40kW / Rack

Computer Room /

Data Center

TRADITIONALFloor-Mount

SUPPLEMENTAL COOLING

+

Traditional Floor-Mount through the first 100-150 w/sqft(or 4-5 kW per rack) and Supplemental Cooling above

that level

Planning for High Density requires A Systems Approach to the Cooling SystemPlanning for High Density requires A Planning for High Density requires A Systems Approach to the Cooling SystemSystems Approach to the Cooling System

Controls Humidity and FiltrationCreates the base airflow distribution

Provides high density sensible cooling at the source

11


Critical Requirements Of The Base Cooling Load EquipmentCritical Requirements Of The Base Critical Requirements Of The Base Cooling Load EquipmentCooling Load Equipment

Cooling units with Variable cooling capacity – DX – Compressors such as the Digital Scroll– VFDs on the fans

Controls so that units can work as a team– Eliminate dehumidification/humidification fighting– Balance the load– Optimize the cooling performance

High efficiency condensers“Green Refrigerant” products

12


Moving the HeatMoving the HeatMoving the Heat

Areas for ImprovementReduced server fan powerHigher temperatures over cooling coils (room temperature / less mixing of hot and cold air)Reduced resistances lower fan power

Room

Rack

Chip

Memory Other componentsNetwork DevicesPower Supplies

Servers

Heat Sink

CRAH

Chiller Heat Rejection

13


Cooling Solutions to Meet the Higher Density RequirementsCooling Solutions to Meet the Cooling Solutions to Meet the Higher Density RequirementsHigher Density Requirements

Requires moving the cooling closer to the heat source to more precisely cool the specific load and not waste energy “brute force” cooling the whole room.Cooling coils may be in multiple locations– External to the Rack – Overhead, Back, Side– In the Rack – Under, Side– Part of the Server

Requires a fluid to be delivered to the cooling coils to transport the heat out of the Data Center– Chilled Water (CW)– Refrigerant (pumped low pressure)

14


Liebert XD & Base Cooling

CoolingUnit

Traditional Cooling OnlyFan Power- 8.5kW per 100 kW of Cooling

Average entering air temperature of 80-84°F

Fan Power- 3.5 kW per 100 kW of Cooling (XD @ 2 kW per 100kW)

Average entering air temperature of 96-98°F

Blower Resistances

Energy Efficiency Benefits for Cooling Closer to the LoadEnergy Efficiency Benefits for Cooling Energy Efficiency Benefits for Cooling Closer to the LoadCloser to the Load

65% less fan powerGreater cooling coil effectiveness

15


Liquid Cooling Configurations –with Building Chilled WaterLiquid Cooling Configurations Liquid Cooling Configurations ––with Building Chilled Waterwith Building Chilled WaterBuilding Chiller XDWP

XDP

CW

RefrigerantCDU

HeatExchanger

Tank

Pump Pump

Valve

2ndLoop

Building Chilled Water

HeatExchanger

Tank

Pump Pump

Valve

2ndLoop

Building Chilled Water

16


Open and Closed ArchitecturesOpen and Closed ArchitecturesOpen and Closed Architectures

Open and Closed Architecture Systems as defined by ASHRAE– The open architecture systems utilize cooling coils

near the heat load either inside or outside the server rack and utilize the room air volume as a thermal storage to ride through short power outages.

– The closed architecture fully encloses the rack with the cooling coil inside. Other provisions are required for power loss ride through.

17


Secondary Fluid ComparisonsSecondary Fluid ComparisonsSecondary Fluid ComparisonsPumped Refrigerant SystemAdvantages– No Water in the DC or

electrical hazards– Micro-channel coil efficiency

(+50%), lower air side pressure drop => lower operating costs

– Smaller piping requirements– Cooling Modules can be

located anywhere– Scalable Capacity (2-3x to CW )

Disadvantages– Small room scalability – Higher Fluid Cost

Chilled Water Based SystemAdvantages– Lowest Fluid Cost– No limitation to room size

Disadvantages– Electrical Hazard– Lower Operating Efficiency– May require fluid treatment to

prevent fouling– Limited overhead cooling

options

18


Flexibility - Extends Your Existing Infrastructure InvestmentFlexibility Flexibility -- Extends Your Existing Extends Your Existing Infrastructure InvestmentInfrastructure Investment

Hot Aisle – Cold Aisle ConfigurationHot Spots, Zones and Hot Rooms

Liebert XDO

On demand, plug-and-play flexibility to add additional capacityCooling at the source of heat with advanced compact heat exchangersMultiple module configurations to meet any Data Center layout Works with any brand of racksCooling fluid is a gas (no water)Self-regulating capacity100% Sensible cooling

Liebert XDV

Liebert XDH

19


Plug and Play Capacity on Demand

20


Liebert XD Energy Efficiency BenefitsLiebert XD Energy Efficiency BenefitsLiebert XD Energy Efficiency BenefitsCooling closer to the source – Dramatically less fan power required to move the air– Higher air temperature entering the coil results in

increase performance

Coil Technology– Microchannel coils – most efficient coil surface

Sensible cooling– All cooling module operate the coil at 5 degree F

above the dew point in the room– Does not unnecessarily dehumidify requiring additional

humidification (value of 7% in efficiency)

21


The Traditional Way (120 racks @ 8kw/rack)The Traditional Way The Traditional Way (120 racks @ 8kw/rack)(120 racks @ 8kw/rack)(12) 30 ton CW Air Handlers

– 10 operational for the load

– 2 stand by

Floor space – 4256 sqft

Requires a raised floor of 48”

PDU PDU PDU PDU PDU PDU

PDU PDUPDU PDU PDU PDU PDU PDU



AC AC AC AC

ACACACAC

ACACACAC

22


Liebert XD Solution (120 racks @ 8kw/rack)Liebert XD Solution Liebert XD Solution (120 racks @ 8kw/rack)(120 racks @ 8kw/rack)

(4) 20 ton CW Air Handlers– 3 operational for the load

– 1 stand by

(6) XDP with (96) XDV– 5/80 operational– 1/16 redundant

Floor Space – 3640 sqft


PDU PDUPDU PDU PDU PDU PDU PDU



AC

AC

AC

AC

XD

XD

XD

XD

XD

XD

23


120 Racks8 kW per Rack

Floor Mount AH

Liebert XD & Floor Mount

Rack Loads 960.0 960.0Fans(BHP) 101.7 44.6Room Latent 5.1 5.1Excess Latent 137.3 29.8PDU 28.8 28.8People 1.5 1.5Build Env 7.9 7.9Lights 5.6 5.6

Total (kw) 1247.9 1083.3


Liebert XD benefits

Total Room Load CalculationsTotal Room Load CalculationsTotal Room Load Calculations

– Less Fan power

– 100% sensible

Smaller load to size the Chiller

24


Summary of EquipmentSummary of EquipmentSummary of EquipmentLiebert XD

(vs traditional method)

– 20% less Chiller plant

– 15% less floor space

– Scalable Platform

Floor Mount AH Liebert XD & Floor Mount

Chillers (3) 250 ton (3) 200 tonCW Pumps (3) 25 HP (3) 20 HP

Floor Mount Units (12) 30 ton (4) 20 tonLiebert XD (96) XDV

Floor Space (sqft) 4256 3640Raised Floor Height (in) 48 24

Scalable Design 8kw 20kw


25


Annual Energy Consumption (kW)Annual Energy Consumption (kW)Annual Energy Consumption (kW)

Cooling closer to the source is more efficient.


Floor Mount AH


Prec Air Units 101.7 25.4XDV 0.0 19.2XDP Pumps 0.0 6.3CW Pumps 42.4 33.9Chiller 195.1 169.4Tower Fans 15.3 13.2Condenser Pumps 15.3 13.2Rehumidification 51.5 11.2

Total Kw 421.3 291.9

Operating Costs (@$.08/kw) 295,231$ 204,571$ Delta (90,660)$

-31%


26


Capital Costs – Total ImpactCapital Costs Capital Costs –– Total ImpactTotal ImpactResults

Traditional – 4256 sqft

Liebert XD – 3640 sqft (-616 ft²)Example at $200 / ft²

Industry range $250-1000 / ft²


Floor Mount AH


Chiller 187,500$ 150,000$ Cooling Units 135,360$ 258,015$ Installation 322,860$ 452,265$ Floor Space -$ (123,200)$ Total (E,I,S) 645,720$ 737,080$

Delta 91,360$

Operating Savings (90,660)$ Payback (yr) 1.0


27


Liebert XD Technology - Fluid Cooling at the Source Drives Down Operating CostsLiebert XD Technology Liebert XD Technology -- Fluid Cooling at Fluid Cooling at the Source Drives Down Operating Coststhe Source Drives Down Operating Costs

Chiller Capacity

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

Traditional CW CRAC CW Enclosed Rack Refrigerant Modules

kW C

hille

r Cap

acity

per

kW

of S

ensi

ble

Hea

t Loa

d Latent LoadFan LoadSensible Load

20% less capacity of the support equipment– Chiller (s)– Cooling Tower / Condensers– Chiller water circulating

pumps– Emergency Generators– Electrical Switch Gear

Annual Power Consumption

0.000

0.050

0.100

0.150

0.200

0.250

0.300

0.350

0.400

0.450

Traditional CW CRAC CW Enclosed Rack Refrigerant Modules

kW p

ower

to c

ool 1

kW

of s

ensi

ble

heat

Fan

Pump (CDU)

Pump (CW)

Chiller

30% Lower

28


Liebert XDFull Range of OpportunitiesLiebert XDLiebert XDFull Range of OpportunitiesFull Range of Opportunities

XDPXDCXDV10

XDH20/32

or

Base Infrastructure

(160 kw)

Future pumping units of larger

capacity

Standard Cooling Modules 10-35kw +++

XDO20

Embedded Cooling

(microchannelintercoolers)

Tested 35-60kw

Developing up to 100kw

Embedded & Chip Cooling

(microchannelintercoolers and

Cooligy chip cooling)Tested 50 kw (100%

redundant)

Capable over 100kw

29


30%0.31

0.2

0.42

0.65

0.36

0

0.1

0.2

0.3

0.4

0.5

0.6

0.719

90

2000

2005

2006

/200

7

2004

/200

6

2006

/200

7

2010

Futu

re

Additional System Opportunities for Improved Cooling Efficiencies Additional System Opportunities for Additional System Opportunities for Improved Cooling Efficiencies Improved Cooling Efficiencies

Traditional Cooling Liebert XD Opportunities

XD M

odul

es

(10

–35

kW

)

Egen

era

(10

–20

kW

)

Embe

dded

Coo

ling

(35

–60

kW

)

Com

pone

ntC

oolin

g(>

50 k

W)

Encl

osed

Rac

k

kw fo

r coo

ling

per k

w o

f ser

ver h

eat l

oad

Data Center Best Practices

Energy Savings driven by reduction in Fan Power (cooling system and server) plus Heat

Transfer efficiency

45%O

ptim

al

30


Cooling Process Throughout The Range Of Rack LoadsCooling Process Throughout The Range Cooling Process Throughout The Range Of Rack LoadsOf Rack Loads

Cooling Process Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Avai

labi

lity

Flex

ibilit

y

TCO

Chip/Component Cooling

Embedded Rack - Refrigerant

Rack Mounted - Refrigerant

Rack Mounted - CW

Traditional CRAH

Rack Load (kW)>0-5 >5-10 >10-15 >15-20 >50>20-25 >25-30 >30-35 >35-50

31


Improving The Total Cost Of Ownership with the Liebert XD Cooling SystemsImproving The Total Cost Of Ownership Improving The Total Cost Of Ownership with the Liebert XD Cooling Systemswith the Liebert XD Cooling Systems

HowCooling closer to the source of the heat makes the heat exchangers more efficient (higher entering air temperatures)Lower total Fan HP

Sensible cooling eliminates wasted energy dehumidify unnecessarily and then having to re-humidifyLess Chiller or DX infrastructure required

Overhead cooling modules require no additional floor spaceCooling solutions that meets the requirements to fill racks of high density serversInfrastructure for modules today and for future server / rack designs

Results in …More cooling capacity for energy consumed

Less Power (energy consumed)Less Power (energy consumed)

Less Power and Capital EquipmentLess Floor Space consumedLess Floor Space consumedExtends your capital life

32


Air Movement

12%

Electricity Transformer/

UPS10%

Lighting, etc.3%

Cooling25%

IT Equipment50%

Lower Power For Cooling Provides More Power For IT EquipmentLower Power For Cooling Provides Lower Power For Cooling Provides More Power For IT EquipmentMore Power For IT Equipment

Data Center Power Draws

59%12%

26%

For the same building power, you can allocate more power to the IT Equipment (18%)

33


Take AwaysTake AwaysTake AwaysCooling solutions for higher density will need to move closer to the load and will require a reliable fluid delivery means– Cost effective cooling solutions exist that can be

employed today that meet future needs– Allow racks to be fully populated

Cooling at the source of the heat load will actually lower your incremental energy consumption– Less power for the cooling system provides more

power for the IT equipment

Results in more available Floor Space and Growth capability for the IT equipment

34


Thank YouContact:

[email protected]

mailto:[email protected]

Date post:	12-Aug-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Future of Cooling High Density Equipment€¦ · Cooling Presents an Opportunity For Energy Savings...

Documents