Jasper Horrell

SKA and Big Data – Up in Space and on the Ground

FUTURES RE-IMAGINED Speaker 14 of 17

Followed by

Theo Priestley

@jasperhorrell

SKA  and  Big  Data  Up  in  Space  

&  on  the  Ground  

Image: Maik Wolleben

KAT-7

Images: SKA SA

MeerKAT

Image: SKA Organization / Swinburne Astronomy Productions

SKA2-Mid

Image: SKA SA

SKA Stations in Africa

Centaurus A in visible light

Image: ESO/WFI

Centaurus A in visible light

Image: ESO/WFI (visible); MPIfR/ESO/APEX/A.Weiss et al. (microwave); NASA/CXC/CfA/R.Kraft et al. (X-ray)

Centaurus A in visible, X-ray (blue) and radio (orange)

Centaurus A in visible, X-ray (blue) and radio (orange)

Double Pulsar

Image: Daniel Cantin, DarwinDimensions, McGill University

Image: SKA Organization / Swinburne Astronomy Productions

SKA Science

Africa (mid-frequency)

Australia (low-frequency & survey) Images: SKA Organization / Swinburne Astronomy Productions

Timeline

Requests for proposal

Proposal responses

Proposal evaluation

Cost ceiling established

Design consortia start

Preliminary design review

Prototype systems deployed

Critical design review

Seek SKA1 funding

Develop SKAO governance

SKA1 construction approved

Tender & procure construction

SKA1 construction

Detailed design of SKA2

SKA1 early science

2013   2014   2015   2016   2017   2018   2019   2020   2021   2022  

Re  base  lining  

3 years  

SKA Project Timeline

Australia  

China  

Italy  

New  Zealand  

Sweden  

India  

Canada  Germany  

Netherlands  

UK  

South  Africa  

SKA Members and Governance

UK Company Limited by Guarantee (may move to a Treaty organization)

Array Telescope

Correlator/ Beamformer/ Pulsar Search

Science Data

Processor Scientists Image: NRAO/AUI

MeerKAT SKA1-Mid SKA2-Mid*

Into Correlator 2 Tbps (2k x office network) (700k x 32 GB / day)

50 Tbps (50k x office network) (17m x 32 GB / day)

up to 5 Pbps (5m x office network) (1.8b x 32 GB / day)

Into Science Processor 0.7 Tbps (240k x 32 GB / day)

20 Tbps (7m x 32 GB / day)

up to 500 Tbps (172m x 32 GB / day)

Into Archive 20 Gbps** (7k x 32 GB / day)

300+ Gbps (100k x 32 GB / day)

up to 2 Tbps (700k x 32 GB / day)

Compute load 200 TFlops 30+ PFlops 3+ EFlops

* Data rates indicative only ** Sustained 32 GB –> large flash drive / mid iPhone / iPod

The (Extreme) Data Challenge

Innovation

Image: SKA SA

ROACH2 - CASPER Collaboration

BDA South

BDA North BDA West

BDA East

National Integrated Cyber Infrastructure System

Storage Network HPC Skills

Multi-disciplinary. Government, Industry (vendor neutral), Academia

16 MW

WCG HPC Initiative

HPC Centre (Big Data Catalyst)

SKA Industry Government Universities CHPC

Whitespace HPC Storage Cloud Network

Green

Public-private platform development

A  li%le  digression  re  on-­‐the-­‐ground  “big  data”  social  innova7on  

….  please  bear  with  me  

This is my bicycle: a “recumbent” Attracts lot’s of comments: “Nice bike!” “Awesome!”

This is my bicycle: a “recumbent” Attracts lot’s of comments: “Nice bike!” “Awesome!” “Mommy, is that man paralysed?”

However,  from  the  “serious  thinkers”:  

“But  why?”  

Fast Comfortable Fun!!

“The  Showcase  Recumbent  Team”    A  racing  team  of  recumbent  bikes  Amateur  riders  Turn  heads,  bring  out  smiles    Promote:  Out-­‐of-­‐the-­‐box  thinking  InnovaTon  Healthy  lifestyle  Be  daring  A  bit  of  fun!  

The  Universe’s  Plan!  

OK…..  What  is  the  Universe  Trying  to  Tell  Me?  J  

✔   ✖  

   GiQ  Meals  

“There  is  no  excuse  for  people  starving.”    “There  is  more  than  enough  to  go  around,  if  only  people  share.”  

   GiQ  Meals  

Cloud-­‐based  InformaTon  System  

✔

✖

✖ ✖

✔ ✔

✔

✖

Skills

Image: SKA SA

E-Learning Project in Carnarvon

Community Knowledge Centre

SKA and Big Data -> Big Opportunity

In Closing

SKA and Big Data -> Big Opportunity Big Data is already happening

In Closing

SKA and Big Data -> Big Opportunity Big Data is already happening It’s not all rocket science! Plenty of societal and business issues to address.

In Closing

SKA and Big Data -> Big Opportunity Big Data is already happening It’s not all rocket science! Plenty of societal and business issues to address. Technology is ‘good enough’. Get started. Develop skills.

In Closing

Collaborate. Big Data is big. There is space. Be bold in your vision. Aim for the stars. Stay grounded. Take it step by step.

Finally