Power considerations for the SKA
Peter Hall
ICRAR/Curtin
Co-Chair, SKA Power Investigation Task Force
AU PEP workshop, Perth, 24 Nov. 2011
Outline
• See conference paper for more technical discussion
• SKA background and power history
• Power Investigation Task Force (PITF)
– Topics and contributions
• Path to SKA power system in pre-construction era
– SKA Project Office and Industry
– Renewable energy: photovoltaic example
• SKA precursor contributions
Background
• SKA is now designing a system, selecting technologies and entering pre-construction)
– SKA1 (10% sensitivity) defined: sparse aperture arrays + dishes
– Advanced instrumentation package defined as possible path to SKA2
• Two candidate central sites: Murchison (WA) and Karoo (RSA)
– 2011, Q1 selection
• Power is a pivotal consideration in system design, and in site selection and development
– Availability and cost determines scientific capability of instrument
– Remote locations, high temperatures, advanced electronics, huge computing
demands, long facility lifetime, …
• Capital and operating dimensions to SKA design + site choice
• Key SKA considerations:
– Low-power telescope design
– Efficient environmental conditioning (passive cooling etc)
– Innovative energy delivery and costing arrangement, e.g.
• Buffer from world-parity energy pricing?
History: SKA power estimates
• Before 2001: blissful ignorance
• c. 2001: estimates of ~10 MW
– Mainly antennas and related systems
• c. 2003: ~20 MW
– Included computing etc.
• 2003-2004: the information explosion!
– (Very) wide fields-of-view to observed and processed
• 2005-2006: scaling from existing installations hundreds of MW panic, denial, …
– “Mr Fusion”
• 2007-present: < 100 MW limit (€ 100M p.a. @ € 0.12 per kWh)
– Pathfinders confront reality
– Capped SKA demands (and capabilities)
– Optimized system design
– Search for innovative funding solutions for power
Power Investigation Task Force (PITF)
• PITF active in refining SKA power estimates – Current numbers of order 30-40 MW for array, 50-60 MW for computing
• Only achieved with substantial innovation
– Recognize demand may be capped and science limited at a given epoch
• Provide starting material for expert pre-construction power infrastructure consultants
• PITF has informed SKA Program Development Office (SPDO) system design
– Incl. new supply technologies: scaling, breakpoints, …
• PITF has promoted information exchange between SKA
Pathfinders, Design Studies, … – Understand and reconcile various power estimates
Recent PITF topics
• Best estimates of SKA power requirements – Capex and opex implications
• Major expenses which help dimension a feasible SKA project
• SKA Pathfinder power technology demonstrations • Trends in generation and supply technologies
• Demand-minimization and trade-offs in major sub-systems, and
environmental conditioning
• Possible solutions to SKA central and remote power needs
– Grid and non-grid
– Fossil and renewable
• Scalability and life-cycle costing of potential power solutions
• Carbon trading and its effect on SKA power optimizations
Mix of information depth – SKA project just entering detailed infrastructure studies
Example: DSP flexibility vs power trade-off
*Plus cost of cooling and delivering power
• ASIC approach:
– 22nm :
2.5 nW/MHz/Gate
> 40 T MACS (4 bit) per device => 25,000 devices
Assuming < 50 % gates switching at any one time: 600kW
Operating cost $600k per annum*
• For a 1018 MACS processing requirement.
• FPGA approach:
– Virtex 6:
2016 x DSP slices clocked at 600 MHz -> 1200 G MACS
~ 25 G MACs per Watt => Requires ~ 106 FPGAS
@ 48 W per device and ~ 48 M Watts for 1018 MACS
Operating cost 1$ per Watt per year => $48M per annum*
Courtesy P. Dewdney
Path to SKA power system
• On entry to pre-construction phase, we need:
– SKA1 design; electronics system design framed to minimize power
– SKA2 physical layouts and representative electrical loads
– Top-level EMC and RFI standards
– Operational model for SKA1, and representative operational model for
SKA2
– Pool of qualified and briefed consultant engineers and suppliers from which
to select prime contractors, sub-contractors and suppliers.
• Default: SKA power generation and transmission infrastructure
will be delivered and operated via agreement with host nation – Possible international industry participation via e.g. equipment supply
contracts
– Variation: large-scale renewable energy collaboration
• Need mechanisms to feed SKA-specific needs to power industry designers and operators
– SKA adds another layer to traditional “electronics – power” divide
Example inner and intermediate layout. 180 dishes on each spiral
arm. 16 AA-hi and 16 AA-lo stations on each spiral arm (not shown).
Dish core: 1500 dishes 5 km diameter.
AA-hi and AA-lo core 167 stations, 5 km diameter.
-10
-8
-6
-4
-2
0
2
4
6
8
10
-10 -8 -6 -4 -2 0 2 4 6 8 10
AAhi core AAlo core Dish core Dishes mid range
SKA 2012 activities • Representative SKA cores • Detailed configuration
analysis • Site specific information
– e.g. grid or non-grid
• Basic load assumptions for dishes and AA stations
– e.g. low / likely / high
• Basic central DSP model • Post-processing computing
located according to successful site submission
• Simple models for remote stations
• Basic operational and lifecycle model
• Brief consultants for more detailed study
Diagram courtesy R. Bolton (U. Camb.)
SKA power in the PEP
• Five work packages defined in Project Execution Plan (PEP)
• WP10.1 Engineering and Management – Coordination activity; led by SKA Project Office (SPO)
• WP10.2 Intra-system Power Design
– Review SKA design proposal with aim to minimize power; led by SPO with strong industry participation; detailed design report addressing light-heavy electrical system interface
• WP10.3 Power System Design
– Major SKA power task; led by industry with SPO input in specialist areas (e.g. EMC); produces complete, costed design for SKA1 and expansion plan for SKA2
• WP10.4 Power System Operation
– Develop and prosecute a plan for SKA power systems operation; led by industry with SPO input; considers power quality, demand evolution, etc.
• WP10.5 Strategic Power Planning
– Assess applicability of emerging power sources and technologies to the SKA; led by industry with SPO input
SKA and power industry - 1
• Current gap between SKA and detailed design mandate needed by industry
• Gap is beginning to close
– Pathfinders are tackling major design issues (e.g. low-RFI supplies)
– 2012 Project activities will bring serious-scale industry interaction
• Superficially SKA is not hugely challenging
– c.f. hundreds of MW at remote natural resources sites in RSA and AU
– Many similarities to supplying a town, its suburbs and outlying areas
• But SKA is idiosyncratic
– 50 year facility lifetime, non-commercial customer
– Geographical diversity
– Dominance of RFI / EMC in power system design
– “Soft” performance - cost targets for remote stations
– Relatively flat load versus time-of-day
– Simultaneous SKA operation and expansion
SKA and power industry - 2
• Need to combine power cost modelling (incl. lifecycle costing) with SKA performance vs cost modelling
– New SKA design models track power requirements
– First-order studies are important; many trade-offs
• e.g. larger built-area vs (more power-hungry) lower Tsys receptors
• Many factors: capex and opex of receptors, DSP, computing ….
• Recognize political aspects of infrastructure provision and operation could confound engineering analysis
• Generic SKA models are invaluable – Allows power experts to begin thinking about important specifics
– Generates key questions for system design and pre-construction
– Dimensions the challenge for governments and funding agencies
• Recognize site-specific issues will rapidly dominate from 2012 – Possibly: grid + renewables in RSA; islanded gas + renewables in Aust
SKA sites have high solar potential
Courtesy Dr Eicke Weber
Courtesy Dr Eicke Weber
SKA target is < 0.12 €/kWh
Solar PV example: Several other technologies possible
SKA precursor: 1.5 MW supply at MRO
•Modular design using the same construction
technique as the ASKAP correlator building
•Identical RFI mitigation techniques
•Two levels of attenuation in the room design
•Power station located > 1 km from antennas
•All RFI emissions reduced to > 20 dB below
MIL Spec 461F
SKA precursor: Radio-quiet 33 kV feeder in Karoo
• Specialized hardware - Mid-span conductor joints require automatic line splices - Only crimped T-connections and pistol grips allowed to connect conductor at ruling spans - All stay wires are equipped with silicon composite long rod insulator
• Specialized techniques - All fuse links selected with soldered tails to prevent fraying 0 1 2 3 4 5
x 108
30
40
50
60
70
80
90
Frequency (Hz)
Magnitude (
dB
uV
/m)
Frequency spectrum for 4mm on a short line
Background noise
4mm gap
Thorough analysis of conventional lines
Solar power for SKA-low?
Courtesy Budi Juswardy
PV panels and EMC
Where does the radiation come from?
Data centre power - example
20
CRAC: Computer Room Air Conditioner
Liam Newcombe “Energy Efficient Data Centres” British Computer Society Data Centre Specialist Group
http://dcsg.bcs.org//component/option,com_docman/task,cat_view/gid,17/
SKA Data Centre Cooling Demonstration
Courtesy Dr Klaus Regenauer-Liebe
Summary
• SKA faces both power demand and supply challenges
• Only achieve < 100 MW with substantial innovation
– Gains from industry trends (e.g. computing)
– Much SKA development needed (e.g. system design, receptor cooling)
• 100 MW € 100M p.a.
– Large component of SKA opex
– Likely limits science re-investment capacity
• Politics and reality of renewable energy could be favourable to
SKA
– But present SKA budget does not allow Project alone to champion this
• SKA capacity will evolve, even for fixed power ceiling
– Reflect in science design reference mission
• Power is a high-stakes game
– Technical and funding advances translate directly into science performance
and capacity