+ All Categories
Home > Documents > Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial...

Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial...

Date post: 21-Feb-2020
Category:
Upload: others
View: 4 times
Download: 0 times
Share this document with a friend
54
1 Grid Computing Dr Simon See High Performance Computing Sun Microsystems Inc
Transcript
Page 1: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

1

Gri

d C

ompu

ting

Dr S

imon

See

Hig

h Pe

rfor

man

ce C

ompu

ting

Sun

Mic

rosy

stem

s Inc

Page 2: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

2

“The

grid

is a

n em

ergi

ng in

frast

ruct

ure

that

will

fund

amen

tally

cha

nge

the

way

w

e th

ink

abou

t – a

nd u

se –

com

putin

g.

The

grid

will

... c

reat

e a

univ

ersa

l sou

rce

of c

ompu

ting

pow

er.”

The

Grid

BLU

EP

RIN

T F

OR

A N

EW

CO

MP

UT

ING

IN

FR

AS

TR

UC

TU

RE

Page 3: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

3

Grid

Com

putin

g E

nviro

nmen

ts:

●H

isto

rical

Per

spec

tive

●In

dust

ry C

hara

cter

istic

s●

Req

uire

men

ts a

nd P

olic

ies

●A

rchi

tect

ure

●Im

plem

enta

tion

●U

sabi

lity

and

Man

agem

ent

●E

xten

sibi

lity

and

Web

Ser

vice

s

AR

CH

ITE

CT

UR

E,

IMP

LEM

EN

TA

TIO

N A

ND

M

AN

AG

EM

EN

T

Page 4: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

4

Def

initi

ons

A G

rid is

a h

ardw

are/

softw

are/

inst

rum

ents

/ser

vice

infra

stru

ctur

e th

at

prov

ides

dep

enda

ble,

con

sist

ent,

perv

asiv

e, in

expe

nsiv

e *

acce

ss to

com

puta

tiona

l and

dat

a ca

pabi

litie

s.

* Fo

ster

, Kes

selm

an, 1

999

His

toric

al P

ersp

ectiv

e

Com

pute

Far

mC

ompu

te F

arm

Grid

Com

putin

g E

nviro

nmen

ts

Gri

dG

rid

Clu

ster

Clu

ster

Har

dwar

e

Inst

rum

ents

Sof

twar

e

Ser

vice

s

Pro

cess

Page 5: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

5

Def

initi

ons

His

toric

al P

ersp

ectiv

e

Gri

d C

ompu

ting

Env

iron

men

ts●

Dis

tribu

ted

Res

ourc

e M

anag

emen

t–

SG

E–

LSF

–N

QS

–D

JM

●P

aral

lel

Com

putin

g E

nviro

nmen

ts–

PV

M–

MP

L–

MP

I–

FOR

TRA

N 9

0–

Lind

a

Page 6: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

6

Grid

Com

putin

g G

enea

logy

●E

arly

Grid

Tec

hnol

ogie

s–

Dis

tribu

ted

Job

Man

ager

; DJM

Net

wor

k Q

ueui

ng S

yste

m:

NQ

S–

Uni

vers

ity R

esea

rch

proj

ects

●M

atur

e C

omm

erci

al P

rodu

cts

–S

un G

rid E

ngin

e (S

un, f

orm

erly

C

odin

e/G

RD

)–

Load

Sha

ring

Faci

lity

(Pla

tform

C

ompu

ting)

–Lo

ad L

evel

er

●In

dust

ry C

olla

bora

tion

/ Le

ader

ship

–G

lobu

s–

Glo

bal G

rid F

orum

–P

latfo

rm G

lobu

s

His

toric

al P

ersp

ectiv

e

Load

Bal

ance

r

Uto

pia

NQ

S

DJM

Task

Bro

ker

Con

dor

DN

QS

NC

Too

lset

LSF

NQ

S/E

xec

NQ

EP

BS

CO

NN

EC

T: Q

ueue

Uni

JES

Load

Lev

eler

Cod

ine

DQ

S

SG

E

SQ

SA

vaki

GG

F

Glo

bus

Pla

tform

Glo

bus

GR

DS

GE

/EE

Page 7: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

7

Indu

stry

Cha

ract

eris

tics

His

toric

al P

ersp

ectiv

e

Life

S

cien

ces

Fina

ncia

l S

ervi

ces

Ele

ctro

nic

Des

ign

Hig

h P

erfo

rman

ce

Com

putin

g

● S

tart

up a

nd c

ost e

ffic

ient

● C

usto

m r

esea

rch

or li

mite

d us

e ap

plic

atio

ns● M

ulti-

day

appl

icat

ion

runs

(BLA

ST)

● E

xpon

entia

l Com

bina

tions

● L

imite

d ad

min

istr

ativ

e st

aff

● C

ompl

emen

tary

tech

niqu

es

● T

ime

to M

arke

t● F

aste

st p

latf

orm

s, la

rges

t Gri

ds● L

icen

se M

anag

emen

t● W

ell e

stab

lishe

d ap

plic

atio

n su

ite● L

arge

lega

cy in

vest

men

t● P

latf

orm

Ow

ners

hip

issu

es

● M

arke

t sim

ulat

ions

● T

ime

IS M

oney

● P

ropr

ieta

ry a

pplic

atio

ns● M

ultip

le P

latf

orm

s● M

ultip

le s

cena

rio

exec

utio

n● N

eed

inst

ant r

esul

ts a

nd a

naly

sis

tool

s

● P

aral

lel R

eser

voir

Sim

ulat

ions

● G

eoph

ysic

al R

ay T

raci

ng● C

usto

m in

-hou

se c

odes

● W

ell e

stab

lishe

d ap

plic

atio

n su

ite

Page 8: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

8

Grid

Evo

lutio

nary

Stra

tegy

:st

artin

g w

ith th

e cl

uste

r grid

Clu

ster

Grid

Dep

artm

enta

l Com

putin

g

• S

impl

est G

rid d

eplo

ymen

t

• M

axim

um u

tiliz

atio

n of

de

partm

enta

l res

ourc

es

• R

esou

rces

allo

cate

d ba

sed

on p

riorit

ies

Ent

erpr

ise

Grid

Ent

erpr

ise

Com

putin

g

• R

esou

rces

sha

red

with

in th

e en

terp

rise

• P

olic

ies

ensu

re c

ompu

ting

on

dem

and

• G

ives

mul

tiple

gro

ups

seam

less

acc

ess

to e

nter

pris

e re

sour

ces

Glo

bal G

rid

Inte

rnet

Com

putin

g

• R

esou

rces

sha

red

over

the

Inte

rnet

• G

loba

l vie

w o

f dis

tribu

ted

data

sets

Gh

hf

iG

id

His

toric

al P

ersp

ectiv

e

Page 9: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

9Exa

mpl

e G

rid C

omm

uniti

es

Virtu

al D

ata

Tool

sR

eque

st P

lann

ing

and

Sche

dulin

g To

ols

Req

uest

Exe

cutio

n M

anag

emen

t Too

ls

Tran

sfor

ms

Dis

tribu

ted

reso

urce

s(c

ode,

sto

rage

,co

mpu

ters

, and

net

wor

k)

Res

ourc

e M

anag

emen

t Se

rvic

es

Res

ourc

e M

anag

emen

t Se

rvic

es

Secu

rity

and

Polic

y Se

rvic

es

Secu

rity

and

Polic

y Se

rvic

es

Oth

er G

rid

Serv

ices

Oth

er G

rid

Serv

ices

Inte

ract

ive

Use

r Too

ls

Prod

uctio

n Te

am

Indi

vidu

al In

vest

igat

orO

ther

Use

rs

Raw

dat

a so

urce

•GriP

hyN

and

Eur

opea

n D

ataG

ridEn

able

(tho

usan

ds o

f) ph

ysic

ists

to p

ool d

ata

& co

mpu

ting

reso

urce

s fo

r da

ta-in

tens

ive

anal

yses

http

://w

ww

.grip

hyn.

org

http

://w

ww

.eu-

data

grid

.org

/

Page 10: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

10

DO

E X

-ray

gra

nd c

hal

lenge:

AN

L, U

SC/I

SI,

NIS

T,

U.C

hic

ago

tom

ogra

phic

rec

onst

ruct

ion

real

-tim

eco

llect

ion

wid

e-ar

eadis

sem

inat

ion

des

ktop &

VR c

lients

w

ith s

har

ed c

ontr

ols

Adva

nce

d P

hoto

n S

ourc

e

Onl

ine

Acc

ess

to

Sci

entif

ic In

stru

men

ts

arch

ival

st

ora

ge

Page 11: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

11

Am

bien

t mic

(tabl

etop

)

Pre

sent

er

mic

Pre

sent

er

cam

era

Aud

ienc

e ca

mer

a

Exa

mpl

e G

rid C

omm

uniti

es●

Acc

ess

Grid

C

olla

bora

tion

–E

nabl

e co

llabo

rativ

e w

ork

at d

ozen

s of

site

s w

orld

wid

e, w

ith s

trong

se

nse

of s

hare

d pr

esen

ce–

Com

bina

tion

of

com

mod

ity a

udio

/vid

eo

tech

+ G

rid te

chno

logi

es

for s

ecur

ity, d

isco

very

, et

c.–

40+

site

s w

orld

wid

e,

num

ber r

isin

g ra

pidl

y

http

://w

ww

.acc

essg

rid.o

rg

Page 12: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

12

The

Grid

Pro

blem

●Fl

exib

le, s

ecur

e, c

oord

inat

ed re

sour

ce

shar

ing

amon

g dy

nam

ic c

olle

ctio

ns o

f in

divi

dual

s, in

stitu

tions

, and

reso

urce

From

“The

Ana

tom

y of

the

Grid

: Ena

blin

g S

cala

ble

Virt

ual O

rgan

izat

ions

●E

nabl

e co

mm

uniti

es (“

virtu

al

orga

niza

tions

”) to

sha

re g

eogr

aphi

cally

di

strib

uted

reso

urce

s as

they

pur

sue

com

mon

goa

ls --

ass

umin

g th

e ab

senc

e of

…–

cent

ral l

ocat

ion,

–ce

ntra

l con

trol,

–om

nisc

ienc

e,

–ex

istin

g tru

st re

latio

nshi

ps.

Page 13: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

13

Ele

men

ts o

f the

Pro

blem

●R

esou

rce

shar

ing

–C

ompu

ters

, sto

rage

, sen

sors

, net

wor

ks, …

–S

harin

g al

way

s co

nditi

onal

: iss

ues

of tr

ust,

polic

y,

nego

tiatio

n, p

aym

ent,

…●

Coo

rdin

ated

pro

blem

sol

ving

–B

eyon

d cl

ient

-ser

ver:

dist

ribut

ed d

ata

anal

ysis

, co

mpu

tatio

n, c

olla

bora

tion,

…●

Dyn

amic

, mul

ti-in

stitu

tiona

l virt

ual o

rgs

–C

omm

unity

ove

rlays

on

clas

sic

org

stru

ctur

es–

Larg

e or

sm

all,

stat

ic o

r dyn

amic

Page 14: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

14

Imag

e co

urt

esy

Har

vey

New

man

, Cal

tech

Dat

a G

rids

for

Hig

h E

nerg

y P

hysi

cs

Tier

2 Ce

ntre

~1

TIP

S

Onl

ine

Syst

em

Off

line

Proc

esso

r Fa

rm

~20

TIPS

CERN

Com

pute

r Ce

ntre

Ferm

iLab

~4

TIPS

Fran

ce R

egio

nal

Cent

re

Ita

ly R

egio

nal

Cent

re

Germ

any

Regi

onal

Ce

ntre

Inst

itut

eIn

stit

ute

Inst

itut

eIn

stit

ute

~0.2

5TIP

S

Phys

icis

t wo

rkst

atio

ns

~100

MBy

tes/

sec

~100

MBy

tes/

sec

~622

Mbi

ts/s

ec

~1 M

Byte

s/se

c

Ther

e is

a “b

unch

cro

ssin

g” e

very

25

nsec

s.Th

ere

are

100

“tri

gger

s” p

er s

econ

dEa

ch t

rigg

ered

eve

nt is

~1

MBy

te in

siz

e

Phys

icis

ts w

ork

on a

naly

sis

“cha

nnel

s”.

Each

inst

itut

e wi

ll ha

ve ~

10 p

hysi

cist

s wo

rkin

g on

one

or

mor

e ch

anne

ls; d

ata

for

thes

e ch

anne

ls s

houl

d be

cac

hed

by t

he

inst

itut

e se

rver

Phys

ics

data

cac

he

~PBy

tes/

sec

~

622

Mbi

ts/s

ec

o

r A

ir F

reig

ht

(dep

reca

ted)

Tier

2 Ce

ntre

~1

TIP

STi

er2

Cent

re

~1 T

IPS

Tier

2 Ce

ntre

~1

TIP

SCa

ltec

h

~1

TIPS

~622

Mbi

ts/s

ec

Tier

Ti

er

00Ti

er

Tier

11

Tier

Ti

er

22

Tier

Ti

er

44

1 TI

PS is

app

roxi

mat

ely

25,0

00

Spec

Int9

5 eq

uiva

lent

s

Page 15: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

15Mat

hem

atic

ians

Sol

ve N

UG

30●

Look

ing

for t

he s

olut

ion

to th

e N

UG

30 q

uadr

atic

as

sign

men

t pro

blem

An

info

rmal

col

labo

ratio

n of

mat

hem

atic

ians

and

co

mpu

ter s

cien

tists

●C

ondo

r-G

del

iver

ed

3.46

E8

CP

U s

econ

ds in

7

days

(pea

k 10

09

proc

esso

rs) i

n U

.S. a

nd

Italy

(8 s

ites)

14,5

,28,2

4,1

,3,1

6,1

5,

10,9

,21,2

,4,2

9,2

5,2

2,

13,2

6,1

7,3

0,6

,20,1

9,

8,1

8,7

,27,1

2,1

1,2

3

Met

aNEO

S:

Arg

onne,

Iow

a, N

ort

hw

este

rn,

Wis

consi

n

Page 16: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

16Net

wor

k fo

r Ear

thqu

ake

Eng

inee

ring

Sim

ulat

ion

●N

EE

Sgr

id: U

S n

atio

nal

infra

stru

ctur

e to

cou

ple

earth

quak

e en

gine

ers

with

exp

erim

enta

l fa

cilit

ies,

dat

abas

es,

com

pute

rs, &

eac

h ot

her

●O

n-de

man

d ac

cess

to

expe

rimen

ts, d

ata

stre

ams,

com

putin

g,

arch

ives

, col

labo

ratio

n

NEESgri

d:

Arg

onne,

Mic

hig

an,

NCSA,

UIU

C,

USC

Page 17: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

17

•C

omm

unity

=–

1000

s of h

ome

com

pute

r us

ers

–Ph

ilant

hrop

ic c

ompu

ting

vend

or (E

ntro

pia)

–R

esea

rch

grou

p (S

crip

ps)

•C

omm

on g

oal=

adv

ance

A

IDS

rese

arch

Hom

e C

ompu

ters

Eva

luat

e A

IDS

Dru

gs

Page 18: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

18

Bro

ader

Con

text

●“G

rid C

ompu

ting”

has

muc

h in

com

mon

w

ith m

ajor

indu

stria

l thr

usts

–B

usin

ess-

to-b

usin

ess,

Pee

r-to

-pee

r, A

pplic

atio

n S

ervi

ce P

rovi

ders

, Sto

rage

Ser

vice

Pro

vide

rs,

Dis

tribu

ted

Com

putin

g, In

tern

et C

ompu

ting…

●S

harin

g is

sues

not

ade

quat

ely

addr

esse

d by

exi

stin

g te

chno

logi

es

–C

ompl

icat

ed re

quire

men

ts: “

run

prog

ram

X a

t site

Y

subj

ect t

o co

mm

unity

pol

icy

P, p

rovi

ding

acc

ess

to d

ata

at Z

acc

ordi

ng to

pol

icy

Q”

–H

igh

perfo

rman

ce: u

niqu

e de

man

ds o

f adv

ance

d &

hi

gh-p

erfo

rman

ce s

yste

ms

Page 19: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

19

Why

Now

?●

Moo

re’s

law

impr

ovem

ents

in c

ompu

ting

prod

uce

high

ly fu

nctio

nal e

ndsy

stem

s●

The

Inte

rnet

and

bur

geon

ing

wire

d an

d w

irele

ss p

rovi

de u

nive

rsal

con

nect

ivity

●C

hang

ing

mod

es o

f wor

king

and

pr

oble

m s

olvi

ng e

mph

asiz

e te

amw

ork,

co

mpu

tatio

n●

Net

wor

k ex

pone

ntia

ls p

rodu

ce d

ram

atic

ch

ange

s in

geo

met

ry a

nd g

eogr

aphy

Page 20: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

20

Net

wor

k E

xpon

entia

ls●

Net

wor

k vs

. com

pute

r per

form

ance

–C

ompu

ter s

peed

dou

bles

eve

ry 1

8 m

onth

s–

Net

wor

k sp

eed

doub

les

ever

y 9

mon

ths

–D

iffer

ence

= o

rder

of m

agni

tude

per

5 y

ears

●19

86 to

200

0–

Com

pute

rs: x

500

–N

etw

orks

: x 3

40,0

00●

2001

to 2

010

–C

ompu

ters

: x 6

0–

Net

wor

ks: x

400

0M

oore

’s L

aw v

s. s

tora

ge impr

ovem

ents

vs.

opt

ical impr

ovem

ents

. Gr

aph

from

Scien

tific

Amer

ican

(J

an-2

001)

by

Cleo

Vile

tt, s

ourc

e Vi

ned

Khos

lan,

Kle

iner

, Cau

fiel

d an

d Pe

rkin

s.

Page 21: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

21

Bus

ines

s D

river

s

●R

educ

ed C

osts

●E

ffici

ent u

se o

f res

ourc

es, T

CO

, RO

I●

Util

ize

late

nt c

ycle

s

●S

horte

n Ti

me

to M

arke

t●

Impr

ove

busi

ness

dec

isio

ns, r

educ

e ris

k●

Fast

er, B

ette

r, C

heap

er●

New

Cap

abili

ties,

impr

oved

qua

lity,

in

nova

tion

Req

uire

men

ts a

nd P

olic

ies

Page 22: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

22

App

licat

ion

Req

uire

men

ts

●B

asic

Info

rmat

ion

–P

rimar

y st

akeh

olde

rs–

Ope

ratin

g S

yste

ms

and

supp

ort m

echa

nism

s–

Dep

loym

ent m

odel

–I/O

and

dat

a re

quire

men

ts●

Exe

cutio

n P

rofil

e–

Dyn

amic

mem

ory

allo

catio

n, fo

otpr

int

–P

erfo

rman

ce c

hara

cter

istic

s, re

fere

nce

benc

hmar

ks–

Net

wor

k I/O

–Jo

b du

ratio

n ch

arac

teris

tics

–U

tiliz

atio

n tre

nds

●A

pplic

atio

n C

hara

cter

istic

s–

Dep

ende

ncie

s–

Prio

ritie

s vs

. oth

er a

pplic

atio

ns–

Dat

a M

anag

emen

t pos

t exe

cutio

n–

Che

ckpo

intin

g ca

pabl

e

Req

uire

men

ts a

nd P

olic

ies

Page 23: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

23

App

licat

ion

Sta

ckA

rchi

tect

ure

Acc

ess

and

Req

uest

Ser

vice

s

Grid

Ser

vice

sN

etw

ork

Ser

vice

sN

FSD

HC

PLD

AP

Iden

tity

NIS

App

licat

ion

Exe

cutio

n S

ervi

ces

Dat

a S

ervi

ces

BLA

ST

NA

STR

AN

MA

KE

Ver

ilog

Page 24: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

24

Arc

hite

ctur

al R

equi

rem

ents

●A

vaila

bilit

y–

Dow

ntim

e im

pact

of G

rid e

nviro

nmen

t–

Impa

ct o

f int

erru

ptio

n of

indi

vidu

al jo

bs–

Acc

epta

ble

mai

nten

ance

win

dow

s●

Sca

labi

lity

–A

ntic

ipat

ed g

row

th o

ver 1

–3–5

yea

rs–

Des

ired

scal

ing

stra

tegy

and

resp

onse

to p

eak

load

s–

Stra

tegy

for t

echn

olog

y re

fresh

, evo

lutio

n●

Man

agea

bilit

y–

Ski

ll se

t/wor

kloa

d of

adm

inis

tratio

n pe

rson

nel

–E

xpec

ted

stab

ility

of a

pplic

atio

ns–

Cod

e m

anag

emen

t, so

ftwar

e di

strib

utio

n m

echa

nism

s●

Sec

urity

–U

ser a

uthe

ntic

atio

n m

echa

nism

s–

Inte

rnet

acc

ess

to g

rid–

Dat

a S

ecur

ity re

quire

men

ts

Req

uire

men

ts a

nd P

olic

ies

Page 25: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

25

Grid

Pol

icy

Man

agem

ent

Req

uire

men

ts a

nd P

olic

ies

Dep

artm

ent

1D

epar

tmen

t 2

Dep

artm

ent 3 D

epar

tmen

t res

ourc

e ac

cess

Cam

pus

wid

e re

sour

ce d

eman

d

Pro

ject

ATeam

B

Con

tract

or X

Pro

ject

C

Use

r 1

Use

r 2

Dep

artm

ent 4

Dep

artm

ent 5

Cle

ar d

efin

ition

and

doc

umen

tatio

n of

prio

ritie

s

and

polic

ies

is k

ey to

Grid

reso

urce

man

agem

ent

Page 26: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

26

Ava

ilabi

lity

Mod

elA

rchi

tect

ure Lo

cal A

rea

Use

rs

Acc

ess

Tier

Com

mod

ity S

ervi

ces

No

Hig

h A

vaila

bilit

y

Res

tart

thro

ugh

SG

E

Pre

miu

m S

ervi

ces

Sun

™ C

lust

er 3

.0

and

RA

S A

rchi

tect

ure

Sun

Grid

Eng

ine

Sun

Clu

ster

age

nt

or S

hado

w M

aste

rN

etw

ork

Ser

vice

s

Sun

Clu

ster

3.0

for N

FS

LDA

P, D

NS

, NIS

;

Mas

ter/S

lave

DH

CP

– n

one

Dat

abas

e S

ervi

ces

Sun

Clu

ster

3.0

Inte

rnet

-Bas

ed

Use

rs

Bac

kup

and

Rec

over

y

Sun

Clu

ster

3.0

for N

etB

acku

p

Net

wor

k S

witc

h R

edun

danc

y

HS

RP

, VR

RP

Sun

Clu

ster

3.0

for P

orta

l

Page 27: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

27

Dat

a M

odel

●N

etw

ork

File

Sys

tem

–D

omin

ates

Grid

E

nviro

nmen

ts–

Nam

ing

Ser

vice

–au

tom

ount

er

●H

ighl

y av

aila

ble

and

scal

able

●K

ey to

effi

cien

t dep

loym

ent

and

man

agem

ent

●S

AN

Tec

hnol

ogie

s us

ually

pr

esen

t beh

ind

data

se

rvic

e●

Loca

l cac

hein

g an

d da

ta

man

agem

ent m

ay p

lay

a ro

le

Arc

hite

ctur

e

/glo

bal

proj

ect

/dat

a

com

mon

/hom

e

usr4

usr3

usr2

usr1

/app

s

/BLA

ST

/am

ber

/SG

E

imag

e

/Lin

ux/S

olar

is

9

/Sol

aris

2.6

apps

/loca

l

scra

tch

Page 28: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

28

Sca

labi

lity

●N

etw

ork

arch

itect

ure

cons

train

s gr

id

scal

abili

ty●

NFS

Arc

hite

ctur

e sc

ales

w/ s

uppo

rt of

SA

N +

Ser

ver T

echn

olog

ies

●S

cala

bilit

y M

odel

●N

etw

ork

Arc

hite

ctur

e●

Sto

rage

Arc

hite

ctur

e●

Ope

ratio

ns M

odel

Arc

hite

ctur

e

Page 29: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

29

Sec

urity

●G

rid S

ecur

ity la

ys o

n to

p of

exi

stin

g S

ecur

ity

arch

itect

ure,

goo

d or

bad

●S

ecur

ity Is

sues

●G

rid m

ay c

onta

ins

reso

urce

s ow

ned

by d

iffer

ent

dist

rust

ful o

rgan

izat

ions

●D

ata

need

s to

get

in/o

ut o

f the

grid

●U

sers

nee

d ac

cess

to th

e gr

id●

Inse

cure

pro

toco

ls s

uch

as N

FS a

nd a

utom

ount

●S

ecur

ity R

equi

rem

ents

●S

ingl

e si

gn-o

n●

Pro

tect

ion

of C

rede

ntia

ls●

Inte

rope

rabi

lity

w/ l

ocal

sec

urity

sta

ndar

ds●

Sec

ure

com

mun

icat

ion

●M

ultip

le im

plem

enta

tions

Arc

hite

ctur

e

Page 30: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

30

Sec

urity

Rec

omm

enda

tions

●U

nder

stan

d S

ecur

ity P

olic

y●

Mul

tiple

Tru

st d

omai

ns, p

olic

y do

mai

ns●

Acc

ess

from

em

ploy

ees,

par

tner

s, c

usto

mer

s●

Sen

sitiv

ity o

f dat

a, in

form

atio

n, re

sour

ces,

com

mun

icat

ion

●S

ecur

e th

e E

nviro

nmen

t●

Sec

urity

arc

hite

ctur

e te

chni

ques

–A

pply

to G

rid S

ervi

ce In

frast

ruct

ure

–S

egre

gate

the

Grid

from

ext

erna

l net

wor

ks w

ith F

irew

alls

–H

arde

n op

erat

ing

syst

em o

n G

rid N

odes

●A

uthe

ntic

atio

n / A

utho

rizat

ion

–E

nfor

ce a

cces

s to

the

Grid

–A

utho

rizat

ion

at th

e us

er/g

roup

leve

l●

Do

not a

llow

use

rs d

irect

con

nect

ion

to th

e gr

id n

odes

●D

evel

op s

ecur

e da

ta tr

ansf

er

Arc

hite

ctur

e

Page 31: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

31

Add

ition

al R

equi

rem

ents

●D

ata

Dis

tribu

tion

●Lo

catio

n, v

olum

e, re

fresh

, and

sec

urity

of d

ata

●U

sabi

lity

●S

kill

set a

nd c

lient

env

ironm

ent c

an im

pact

us

abili

ty o

f env

ironm

ent

●P

sych

olog

ical

fact

ors

impo

rtant

●O

pera

tions

Man

agem

ent

●Li

mite

d st

aff t

o m

anag

e hu

ndre

ds o

r tho

usan

ds

of C

PU

s●

Res

ourc

es a

dded

in la

rge

bloc

ks●

Cha

nge

cont

rol i

s ex

trem

ely

criti

cal o

pera

tion

Req

uire

men

ts a

nd P

olic

ies

Page 32: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

●R

esou

rce

man

agem

ent s

oftw

are

that

ag

greg

ates

com

pute

pow

er a

nd d

eliv

ers

it as

a n

etw

ork

serv

ice.

●P

rovi

des

cons

iste

nt a

nd p

erva

sive

ac

cess

to h

igh-

thro

ughp

ut c

ompu

tatio

nal

capa

bilit

ies,

and

the

job

acco

untin

g an

d st

atis

tics

need

ed to

mon

itor a

nd im

prov

e re

sour

ce u

tiliz

atio

n.

Grid

Eng

ine

Page 33: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

33

Grid

Evo

lutio

nary

Stra

tegy

:st

artin

g w

ith th

e cl

uste

r grid

Clu

ster

Grid

Dep

artm

enta

l Com

putin

g

• S

impl

est G

rid d

eplo

ymen

t

• M

axim

um u

tiliz

atio

n of

de

partm

enta

l res

ourc

es

• R

esou

rces

allo

cate

d ba

sed

on p

riorit

ies

Ent

erpr

ise

Grid

Ent

erpr

ise

Com

putin

g

• R

esou

rces

sha

red

with

in th

e en

terp

rise

• P

olic

ies

ensu

re c

ompu

ting

on

dem

and

• G

ives

mul

tiple

gro

ups

seam

less

acc

ess

to e

nter

pris

e re

sour

ces

Glo

bal G

rid

Inte

rnet

Com

putin

g

• R

esou

rces

sha

red

over

the

Inte

rnet

• G

loba

l vie

w o

f dis

tribu

ted

data

sets

Gh

hf

iG

id

His

toric

al P

ersp

ectiv

e

Page 34: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

34

Ope

n G

rid S

ervi

ces

Arc

hite

ctur

e (O

GS

A)

●E

volu

tion

and

Inte

grat

ion

of G

rid te

chno

logi

es a

nd W

eb

Ser

vice

s●

OG

SA

def

ines

a “G

rid S

ervi

ce”

●In

term

s of

WS

DL

inte

rface

s, d

efin

es m

echa

nism

s:●

crea

ting

dist

ribut

ed

syst

ems

●lif

etim

e m

anag

emen

t●

chan

ge m

anag

emen

t

●cr

eden

tial m

anag

emen

t●

notif

icat

ion

●N

otifi

catio

n●

Aut

horiz

atio

n●

Ser

vice

Cre

atio

n●

Ser

vice

Reg

istry

●M

anag

eabi

lity

●C

oncu

rren

cy

Hos

ting

Env

/runt

ime:

C, J

2EE

Sys

tem

s In

frast

ruct

ure

Grid

Ser

vice

... o

ther

inte

rface

s

Ser

vice

Dat

a

Ele

men

t

Ser

vice

Dat

a

Ele

men

t

Ser

vice

Dat

a

Ele

men

tIm

plem

enta

tion/

Inte

rface

s

Page 35: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

35

Sun Technologies •Ja

va™

•Ji

ni™

•V

irtua

lizat

ion

of n

etw

ork,

co

mpu

te, s

tora

ge

reso

urce

s—

N1

initi

ativ

e•

Res

ourc

e op

timiz

atio

n•

Thin

-nod

e an

d fa

tS

MP

clu

ster

s•

Sin

gle

OS

(mon

olith

ic)

scal

abili

ty•

RA

S•

Dat

a C

ente

r m

anag

ers

•IT

sta

keho

lder

s

Clie

ntP

orta

lS

hare

d P

ool o

f Res

ourc

es

Constituencies

Sun

Col

labo

rativ

e C

ompu

ting

•S

cien

tist d

evel

oper

•E

ngin

eer

deve

lope

r•

Ser

vice

Pro

vide

rs•

ISV

s

•S

cien

tist

•E

ngin

eer

•S

un O

NE

Stu

dio™

S

uite

•S

un O

NE

•Ja

va™

•Ji

ni™

Page 36: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

36

Usa

bilit

y●

Com

man

d lin

e in

tera

ctio

n to

laun

ch jo

bs●

GU

I’s o

ften

live

in U

nix®

w

orld

●A

dditi

onal

ste

ps/p

rodu

cts

inte

grat

ing

with

Win

dow

s O

E –N

FS In

tegr

atio

n–

Teln

et w

indo

ws

–X

Ser

ver

–U

ser I

D M

gt.

●Te

chni

cal C

ompu

ting

Por

tal

–C

entra

lize

Job

Mgt

.–

Sta

ndar

dize

acc

ess

and

auth

entic

atio

n–

Ext

end

stan

dard

inte

rface

to

user

s,cu

stom

ers,

partn

ers

Usa

bilit

y an

d M

anag

emen

t

Page 37: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

37

Sun

Ena

blin

g Te

chno

logi

es

●S

un™

Grid

Eng

ine

Sof

twar

e–

Hig

h-th

roug

hput

com

puta

tiona

l ca

pabi

litie

s an

d re

sour

ce

utili

zatio

n fo

r clu

ster

grid

s–

Ent

erpr

ise

Edi

tion

adds

pol

icy

man

agem

ent f

or s

hare

d-ow

ners

hip

cam

pus

grid

s

●S

un O

NE

™ S

oftw

are

–P

orta

l Ser

ver:

Inte

rnet

ser

vice

s de

ploy

men

t pla

tform

–S

ervi

ces

to q

uick

ly, s

ecur

ely

depl

oy te

chni

cally

dem

andi

ng

porta

ls

Sup

ports

Sol

aris

, Lin

ux,

and

Oth

erO

pera

ting

Sys

tem

s

Page 38: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Wha

t is

a P

orta

l ?●

A W

eb S

erve

r plu

s–

Secu

rity

–Au

then

ticat

ion

(Uni

x, L

DAP

, sec

ured

,...)

–Pe

rson

aliz

atio

n–

Appl

icat

ion

Acce

ss (b

usin

ess

& te

chni

cal)

–C

onte

nt A

ggre

gatio

n

Sun

ON

E P

orta

l Ser

ver©

Page 39: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Wha

t is

an L

S-D

YN

A P

orta

l ?●

LS-D

YN

A a

cces

s th

roug

h a

Bro

wse

r–

Mak

e sy

stem

com

plex

ities

transparent

–R

emot

e Ac

cess

/ Jo

b Su

bmis

sion

–R

emot

e Jo

b M

anag

emen

t–

HTM

L fo

rms

for j

ob s

ubm

issi

on–

Upl

oad

/ Dow

nloa

d fil

es re

mot

ely

–Se

para

te In

terfa

ces

for S

uppl

iers

/ Pa

rtner

s S

ecu

re a

nyt

ime,

an

ywh

ere

acce

ss

Page 40: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

LS-D

YN

A P

orta

l Arc

hite

ctur

e

HPC

Se

rver

s

(HPC

SAN

s)H

PC S

tora

ge

Hig

h Sp

eed

Dat

a Tr

ansf

er

Web

Bro

wse

rsR

emot

e Fi

le

Edit/

Visu

aliz

e/Tr

ansf

er

Sun

ON

E P

orta

l / G

rid

Eng

ine

Page 41: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Logi

n S

cree

n

Page 42: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Use

r Acc

ess

Vie

w

Page 43: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Job

Sub

mis

sion

Page 44: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Job

Inpu

t For

m

Page 45: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

File

Vie

wer

Page 46: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

Out

put F

ile

Page 47: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

TCP

Har

dwar

e an

d S

oftw

are

Sta

ck

Intra

net

Por

tal

Wire

less

Por

tal

(futu

re)

Sun

Mgt

Cnt

r

Inte

rnet

Por

tal

Sun

Ser

vers

T3 S

tora

ge R

acks

Sun

TCP

Inte

grat

ion

Tool

s

Tech

nica

l App

licat

ions

(IS

V&

Lega

cy)

Sol

aris

Ope

ratin

g E

nviro

nmen

t

Sun

Grid

Eng

ine

Clu

ster

Tool

s (M

PI)

Sun

ON

E P

orta

l Ser

ver

Page 48: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

With

TC

P, Y

ou C

an...

Enj

oy s

ecur

e an

ytim

e, a

nyw

here

acc

ess

– su

bmit

jobs

rem

otel

y

Mak

e H

PTC

reso

urce

s av

aila

ble

to o

ther

use

rs w

ithou

t dup

licat

ing

the

inve

stm

ent

Upl

oad

files

with

the

clic

k of

a b

utto

n

Im

prov

e re

sour

ce m

anag

emen

t and

har

ness

idle

com

putin

g cy

cles

Man

age

job

subm

issi

on v

ia fo

rms

Dyn

amic

ally

che

ck o

n th

e st

atus

of j

obs

Rec

eive

em

ail n

otifi

catio

n w

hen

jobs

are

com

plet

e

Dow

nloa

d/vi

sual

ize

outp

ut fi

les

rem

otel

y

Eas

ily s

hare

resu

lts w

ith c

olla

bora

tors

Qui

ckly

add

lega

cy a

pps

for s

harin

g an

d re

mot

e us

e

Page 49: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

49

Sun

Pow

ers

the

Grid

•C

aprio

n P

harm

aceu

tical

s•

Cog

nige

n C

orp.

•E

PC

C•

Ford

Mot

or C

ompa

ny•

Ohi

o S

uper

com

pute

r C

ente

r•

Mot

orol

a•

RW

TH A

ache

n

•W

hite

Ros

e in

itiat

ive

Com

pute

C

ompu

te

Farm

s/Fa

rms/

Clu

ster

Grid

sC

lust

er G

rids

Cam

pus/

Cam

pus/

Ent

erpr

ise

Ent

erpr

ise

Grid

sG

rids

Glo

bal G

rids

Tech

nica

l Te

chni

cal

Com

putin

g C

ompu

ting

Por

tal

Por

tal

Page 50: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

50

Ford

Mot

or C

ompa

ny

●S

un H

PTC

sol

utio

n–

500

dual

-CP

U w

orks

tatio

ns–

Sun

Grid

Eng

ine

softw

are

●B

usin

ess

Impa

ct–

Run

MC

AD

app

licat

ions

on

500

CP

Us

by d

ay–

Run

MC

AE

app

licat

ions

7 x

24

on s

ame

syst

ems

conf

igur

ed a

s a

grid

●50

0 C

PU

s da

ytim

e; 1

000

CP

Us

nigh

ts a

nd w

eeke

nds

–G

reat

ly in

crea

sed

thro

ughp

ut–

Mor

e an

alys

is d

rives

hig

her q

ualit

y re

sear

ch a

nd lo

wer

co

st

–Fa

ster

des

ign

deci

sion

s re

duce

tim

e to

mar

ket

Page 51: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

51

Cap

rion

Pha

rmac

eutic

als:

●S

un H

PTC

sol

utio

n–

Bas

ed o

n S

PA

RC

/Sol

aris

–R

ange

of S

un F

ire s

erve

rs (2

80R

to 6

800)

–Fo

rte™

(Sun

Stu

dio)

Dev

elop

men

t Too

ls

–S

un P

rofe

ssio

nal S

ervi

ces

●B

ioin

form

atic

s fir

m, l

eade

r in

prot

eom

ics

–D

isco

vers

and

dev

elop

s ph

arm

aceu

tical

pro

duct

s

●C

riter

ia–

Bui

ld c

ompu

te fa

rm th

at c

an a

ddre

ss th

e bi

gges

t iss

ues

in li

fe

scie

nces

toda

y–

Pro

cess

hun

dred

s of

tera

byte

s of

dat

a pe

r yea

r–

Cre

ate

arch

itect

ure

that

can

gro

w e

xpon

entia

lly–

Bet

ter c

olla

bora

tion,

fast

er ti

me-

to-m

arke

t

LEA

DIN

G P

RO

TE

IN A

NA

LYS

IS C

OM

PU

TE

F

AR

M

Page 52: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

52

Aac

hen

Uni

vers

ity

●Le

adin

g E

urop

ean

tech

nica

l uni

vers

ity–

Rep

lace

out

date

d, v

ecto

r-ba

sed

supe

rcom

pute

r–

Four

tera

flops

goa

l–

Col

labo

ratio

n fo

cus

on C

FD a

nd V

R T

ools

●C

riter

ia–

ISV

rela

tions

hips

Sca

labi

lity,

ava

ilabi

lity

–P

rice/

perfo

rman

ce–

Ven

dor w

ith c

olla

bora

tion-

orie

nted

app

roac

h●

Sun

HP

TC s

olut

ion

–Fo

ur S

un F

ire 1

5K s

yste

ms,

16

Sun

Fire

680

0 sy

stem

s–

Agg

rega

tes

to 1

.2 te

raflo

ps–

Sun

™ H

PC

Clu

ster

Tool

s so

ftwar

e–

Sun

™ G

rid E

ngin

e fo

r sta

tew

ide

acce

ss

Page 53: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

53

HP

C V

irtua

l Lab

s (H

PC

VL)

●C

onso

rtium

of F

our C

anad

ian

Uni

vers

ities

Rec

ently

dep

loye

d m

assi

ve S

un H

PC

env

ironm

ent

–E

xpec

ted

to b

e am

ong

top

100

HP

C s

ites

in th

e w

orld

Crit

eria

–In

nova

tive

partn

ersh

ip p

rogr

am–

Rob

ustn

ess,

relia

bilit

y–

Pric

e/pe

rform

ance

–Fu

ture

pro

duct

road

map

●S

un H

PTC

Sol

utio

n–

10 S

un F

ire™

680

0s a

nd 2

Sun

Fire

15K

s –

Sun

Ent

erpr

ise™

220

R, S

un S

torE

dge™

T3

arra

ys–

Sol

aris

™ 8

Ope

ratin

g E

nviro

nmen

t–

Sun

HP

C C

lust

erTo

ols™

, For

te™

, Sun

Grid

Eng

ine,

Sun

Res

ourc

e M

anag

er™

, Sun

Man

agem

ent C

ente

r, S

olst

ice

Dis

kSui

te™

sof

twar

e

Page 54: Grid Computing - 東京大学 · “Grid Computing” has much in common with major industrial thrusts – Business-to-business, Peer-to-peer, Application Service Providers, Storage

54

Con

clus

ions

Grid

Com

putin

g ha

s th

e po

tent

ial

to e

xten

d co

mpu

ting

into

a u

tility

-ty

pe e

nviro

nmen

t. If

this

is to

ha

ppen

, Grid

com

putin

g m

ust b

e bu

ilt u

pon

an in

frast

ruct

ure

desi

gned

to p

rovi

de c

arrie

r-gr

ade

Qua

lity

of S

ervi

ce


Recommended