A National Level Paper PresentationOn

GRID COMPUTING

AtSPURTHI

Submitted by

B.V. Subrahmanyam J. Sarath Chandra E-Mail: veeramanimail@gmail.com E-Mail: sarath.jan17@gmail.com

Branch of Computer Science and Engineering

Gudlavalleru Engineering CollegeGudlavalleru-521356

1

Abstrac t

Gr id compu t ing , emerg ing a s a new pa rad igm fo r nex t -

gene ra t i on compu t ing , enab l e s t he sha r i ng , s e l e c t i on , and

agg rega t i on o f geog raph i ca l l y d i s t r i bu t ed he t e rogeneous r e sou rce s

fo r so lv ing l a rge - sca l e p rob l ems i n s c i ence , eng inee r i ng , and

commerce . The r e sou rce s i n t he Gr id a r e he t e rogeneous and

geog raph i ca l l y d i s t r i bu t ed . Ava i l ab i l i t y , u sage and cos t po l i c i e s

va ry depend ing on t he pa r t i cu l a r u se r , t ime , p r i o r i t i e s and goa l s .

I t enab l e s t he r egu l a t i on o f supp ly and demand fo r r e sou rce s .

I t p rov ide s an i ncen t i ve fo r r e sou rce owne r s t o pa r t i c i pa t e

i n t he Gr id ; and mo t iva t e s t he u se r s t o t r ade -o f f be tween

dead l i ne , budge t , and t he r equ i r ed l eve l o f qua l i t y o f s e rv i ce . The

t he s i s demons t r a t e s t he c apab i l i t y o f e conomic -ba sed sy s t ems fo r

w ide -a r ea pa r a l l e l and d i s t r i bu t ed compu t ing by deve lop ing u se r s ’

qua l i t y -o f - s e rv i ce r equ i r emen t s -ba sed s chedu l i ng s t r a t eg i e s ,

a l go r i t hms , and sy s t ems . I t demons t r a t e s t he i r e f f ec t i venes s by

pe r fo rming s chedu l i ng expe r imen t s on t he Wor ld -Wide Gr id fo r

so lv ing pa rame te r sweep—task and da t a pa r a l l e l—app l i c a t i ons .

Th i s pape r focuse s on i n t roduc t i on , g r i d de f i n i t i on . I t cove r s

abou t g r i d cha rac t e r i s t i c s , t ype s o f g r i d s and an example

de sc r i b ing a commun i ty g r i d mode l . I t g i ve s an ove rv i ew o f g r i d

t oo l s , va r i ous componen t s , advan t ages fo l l owed by conc lu s ion .

1 . INTRODUCTION:

2

Thi s The Gr id un i t e s s e rve r s and s t o r age i n to a s i ng l e

sy s t em tha t a c t s a s a s i ng l e compu te r - a l l you r app l i c a t i ons t ap

i n to a l l you r compu t ing power . Ha rdware r e sou rce s a r e fu l l y

u t i l i z ed and sp ike s i n demand a r e me t w i th e a se . Th i s Web s i t e

sponso red by Orac l e b r i ngs you t he r e sou rce s you need t o

eva lua t e you r o rgan i za t i on ' s adop t i on o f g r i d t e chno log i e s . The

Gr id i s r e ady when you a r e .

2 . THE GRID:

The Gr id i s t he compu t ing and da t a managemen t

i n f r a s t ruc tu r e t ha t w i l l p rov ide t he e l e c t ron i c unde rp inn ing fo r a

g loba l soc i e ty i n bus ine s s , gove rnmen t , r e s ea r ch , s c i ence and

en t e r t a i nmen t , i n t eg ra t e ne twork ing , commun ica t i on , compu ta t i on

and i n fo rma t ion t o p rov ide a v i r t ua l p l a t fo rm fo r compu ta t i on and

da t a managemen t i n t he s ame way t ha t t he In t e rne t i n t eg ra t e s

r e sou rce s t o fo rm a v i r t ua l p l a t fo rm fo r i n fo rma t ion . The Gr id i s

t he compu t ing and da t a managemen t i n f r a s t ruc tu r e t ha t w i l l

p rov ide t he e l e c t ron i c . Gr id i n f r a s t ruc tu r e w i l l p rov ide u s w i th

t he ab i l i t y t o dynamica l l y l i nk t oge the r r e sou rce s a s an ensemble

t o suppo r t t he execu t i on o f l a rge - sca l e , r e sou rce - i n t ens ive , and

d i s t r i bu t ed app l i c a t i ons .

G r id i s a t ype o f pa r a l l e l and d i s t r i bu t ed sy s t em tha t enab l e s

t he sha r i ng , s e l e c t i on , and agg rega t i on o f geog raph i ca l l y

d i s t r i bu t ed " au tonomous" r e sou rce s dynamica l l y a t r un t ime

depend ing on t he i r ava i l ab i l i t y , c apab i l i t y , pe r fo rmance , co s t , and

u se r s ' qua l i t y -o f - s e rv i ce r equ i r emen t s .

3 . GRID COMPUTING CHARACTERSTICS .

An en t e rp r i s e - compu t ing g r i d i s cha rac t e r i z ed by t h r ee p r ima ry

f ea tu r e s -

Dive r s i t y ;

Decen t r a l i z a t i on ; and

Dynamism

Diver s i t y :

3

A typ i ca l compu t ing g r i d cons i s t s o f many hund reds o f

managed r e sou rce s o f va r i ous k inds i nc lud ing s e rve r s , s t o r age ,

Da t abase Se rve r s , App l i c a t i on Se rve r s , En t e rp r i s e App l i c a t i ons ,

and sy s t em se rv i ce s l i ke D i r ec to ry Se rv i ce s , Secu r i t y and Iden t i t y

Managemen t Se rv i ce s , and o the r s . Manag ing t he se r e sou rce s and

t he i r l i f e cyc l e i s a complex cha l l enge .

Decen t ra l i za t i on :

Trad i t i ona l d i s t r i bu t ed sy s t ems have t yp i ca l l y been managed

f rom a cen t r a l admin i s t r a t i on po in t . A compu t ing g r i d fu r t he r

compounds t he se cha l l enges

s ince t he r e sou rce s c an be even more decen t r a l i z ed and may be

geog raph i ca l l y d i s t r i bu t ed ac ros s many d i f f e r en t da t a c en t e r s

w i th in an en t e rp r i s e .

Dynamism :

Componen t s o f a t r ad i t i ona l app l i c a t i on t yp i ca l l y run i n a

s t a t i c env i ronmen t w i thou t t he need ing t o add re s s r ap id ly

chang ing demands . I n a compu t ing g r i d , howeve r , t he sy s t ems and

app l i c a t i ons need t o be ab l e t o f l ex ib ly adap t t o chang ing

demand . Fo r i n s t ance , w i th t he l a t e b ind ing na tu r e and c ro s s -

p l a t fo rm p rope r t i e s o f web s e rv i ce s , an app l i c a t i on dep loyed on

t he g r i d may cons i s t o f a cons t an t l y chang ing s e t o f componen t s .

A t d i f f e r en t po in t s i n t ime , t he se componen t s c an be hos t ed on

d i f f e r en t nodes i n t he ne twork . Manag ing an app l i c a t i on i n such a

dynamic env i ronmen t c an be a cha l l eng ing unde r t ak ing .

4 . A COMMUNITY OF GRID MODEL:

Over t he l a s t decade , t he Gr id commun i ty ha s begun t o

conve rge on a l aye red mode l t ha t a l l ows deve lopmen t o f t he

complex sy s t em o f s e rv i ce s and so f twa re r equ i r ed t o i n t eg ra t e

Gr id r e sou rce s . The Communi ty Gr id Mode l ( a l aye red abs t r ac t i on

o f t he g r i d ) be ing deve loped i n a l oose ly coo rd ina t ed manne r

t h roughou t a cademia and t he commerc i a l s ec to r .

The bo t t om ho r i zon t a l l aye r o f t he Communi ty Gr id Mode l

cons i s t s o f t he ha rdware r e sou rce s t ha t unde r l i e t he Gr id . Such

r e sou rce s i nc lude compu te r s , ne tworks , da t a a r ch ive s ,

4

i n s t rumen t s , v i sua l i z a t i on dev i ce s and so on . Moreove r , t he

r e sou rce poo l r ep re sen t ed by t h i s l aye r i s h igh ly dynamic , bo th a s

a r e su l t o f new r e sou rce s be ing added t o t he mix and o ld

r e sou rce s be ing r e t i r ed , and a s a r e su l t o f va ry ing obse rvab l e

pe r fo rmance o f t he r e sou rce s i n t he sha red , mu l t i - u se r

env i ronmen t o f t he Gr id .

Figure 1 : Laye red a r ch i t e c tu r e o f t he Communi ty Gr id Mode l .

The nex t ho r i zon t a l l aye r ( common in f r a s t ruc tu r e ) cons i s t s

o f t he so f twa re s e rv i ce s and sy s t ems , wh ich v i r t ua l i z ed t he Gr id .

The key concep t a t t he common in f r a s t ruc tu r e l aye r i s commun i ty

ag reemen t on so f twa re , wh ich w i l l r ep re sen t t he Gr id a s a un i f i ed

v i r t ua l p l a t fo rm and p rov ide t he t a rge t f o r more focused so f twa re

and app l i c a t i ons .

The nex t ho r i zon t a l l aye r ( u se r and app l i c a t i on - focused

Gr id midd l eware , t oo l s and s e rv i ce s ) con t a in s so f twa re packages

bu i l t a t op t he common in f r a s t ruc tu r e . Th i s so f twa re s e rve s t o

enab l e app l i c a t i ons t o more p roduc t i ve ly u se Gr id r e sou rce s by

mask ing some o f t he complex i t y i nvo lved i n sy s t em ac t i v i t i e s

such a s au then t i c a t i on , f i l e t r ans f e r .

5 . TYPES OF GRID

Gr id compu t ing can be u sed i n a va r i e t y o f ways t o add re s s

va r i ous k inds o f app l i c a t i on r equ i r emen t s . Of t en , g r i d s a r e

c a t ego r i zed by t he t ype o f so lu t i ons t ha t t hey be s t add re s s . The

t h r ee p r ima ry t ypes o f g r i d s a r e

5

Computa t i ona l g r id

A compu ta t i ona l g r i d i s f ocused on s e t t i ng a s ide r e sou rce s

spec i f i c a l l y fo r compu t ing power . I n t h i s t ype o f g r i d , mos t o f t he

mach ine s a r e h igh -pe r fo rmance s e rve r s .

Scaveng ing g r id

A s caveng ing g r i d i s mos t common ly u sed w i th l a rge

number s o f de sk top mach ine s . Mach ine s a r e s cavenged fo r

ava i l ab l e CPU cyc l e s and o the r r e sou rce s . Owne r s o f t he de sk top

mach ine s a r e u sua l l y g iven con t ro l ove r when t he i r r e sou rce s a r e

ava i l ab l e t o pa r t i c i pa t e i n t he g r i d .

Data g r id

A da t a g r i d i s r e spons ib l e fo r hous ing and p rov id ing

acce s s t o da t a a c ro s s mu l t i p l e o rgan i za t i ons . Use r s a r e no t

conce rned w i th whe re t h i s da t a i s l oca t ed a s l ong a s t hey have

acce s s t o t he da t a . Fo r example , you may have two un ive r s i t i e s

do ing l i f e s c i ence r e sea r ch , e ach w i th un ique da t a . A da t a g r i d

wou ld a l l ow them to sha re t he i r da t a , manage t he da t a , and

manage s ecu r i t y i s sue s such a s who ha s a cce s s t o wha t da t a .

Ano the r common d i s t r i bu t ed compu t ing mode l t ha t i s o f t en

a s soc i a t ed w i th o r con fused w i th Gr id compu t ing i s pee r - t o -pee r

compu t ing . I n f a c t , some cons ide r t h i s i s ano the r f o rm o f Gr id

compu t ing .

6 . GRID COMPONENTS:A HIGH LEVEL PERSPECTIVE

Depend ing on t he g r i d de s ign and i t s expec t ed u se , some o f

t he se componen t s may o r may no t be r equ i r ed , and i n some ca se s

t hey may be combined t o fo rm a hyb r id componen t .

6 .1 . Por ta l /u se r in t e r face

J u s t a s a consumer s ee s t he power g r i d a s a r e cep t ac l e i n

t he wa l l , a g r i d u se r shou ld no t s ee a l l o f t he complex i t i e s o f t he

compu t ing g r i d . A l though t he u se r i n t e r f ace can come i n many

fo rms and be app l i c a t i on - spec i f i c . A g r i d po r t a l p rov ide s t he

i n t e r f ace fo r a u se r t o l aunch app l i c a t i ons t ha t w i l l u se t he

6

r e sou rce s and s e rv i ce s p rov ided by t he g r i d . F rom th i s

pe r spec t i ve , t he u se r s ee s t he g r i d a s a v i r t ua l compu t ing r e sou rce

j u s t a s t he consumer o f power s ee s t he r e cep t ac l e a s an . i n t e r f ace

t o a v i r t ua l gene ra to r .

F igu re 2 : Pos s ib l e u se r v i ew o f a g r i d

6 .2 . Secur i t y

A ma jo r r equ i r emen t fo r Gr id compu t ing i s s ecu r i t y . A t t he

ba se o f any g r i d env i ronmen t , t he r e mus t be mechan i sms t o

p rov ide s ecu r i t y , i nc lud ing au then t i c a t i on , au tho r i za t i on , da t a

enc ryp t i on , and so on . The Gr id Secu r i t y In f r a s t ruc tu r e (GSI )

componen t o f t he G lobus Too lk i t p rov ide s robus t s ecu r i t y

mechan i sms . The GSI i nc ludes an OpenSSL imp lemen ta t i on . I t

a l so p rov ide s a s i ng l e s i gn -on mechan i sm , so t ha t once a u se r i s

au then t i c a t ed , a p roxy ce r t i f i c a t e i s c r ea t ed and u sed when

pe r fo rming ac t i ons w i th in t he g r i d . When de s ign ing you r g r i d

env i ronmen t , you may use t he GSI s i gn - in t o g r an t a cce s s t o t he

po r t a l , o r you may have you r own secu r i t y fo r t he po r t a l . The

po r t a l w i l l t hen be r e spons ib l e fo r s i gn ing i n t o t he g r i d , e i t he r

u s ing t he u se r ' s c r eden t i a l s o r u s ing a gene r i c s e t o f c r eden t i a l s

f o r a l l au tho r i z ed u se r s o f t he po r t a l .

7

F igu re 3 : Secu r i t y i n a g r i d env i ronmen t

6 .3 . Broker

Once au then t i c a t ed , t he u se r w i l l be l aunch ing an

app l i c a t i on . Based on t he app l i c a t i on , and pos s ib ly on o the r

pa r ame te r s p rov ided by t he u se r , t he nex t s t ep i s t o i den t i fy t he

ava i l ab l e and app rop r i a t e r e sou rce s t o u se w i th in t he g r i d . Th i s

t a sk cou ld be ca r r i ed ou t by a b roke r func t i on . A l though t he r e i s

no b roke r imp lemen ta t i on p rov ided by G lobus , t he r e i s an LDAP-

based i n fo rma t ion s e rv i ce . Th i s s e rv i ce i s c a l l ed t he Gr id

In fo rma t ion Se rv i ce (GIS ) , o r more common ly t he Mon i to r i ng and

D i scove ry Se rv i ce (MDS) . Th i s s e rv i ce p rov ide s i n fo rma t ion

abou t t he ava i l ab l e r e sou rce s w i th in t he g r i d and t he i r s t a t u s . A

b roke r s e rv i ce cou ld be deve loped t ha t u t i l i z e s MDS.

F igu re 4 : B roke r s e rv i ce

8

6 .4 . Schedu le r

Once t he r e sou rce s have been i den t i f i ed , t he nex t l og i ca l

s t ep i s t o s chedu l e t he i nd iv idua l j obs t o run on t hem. I f s e t s o f

s t and -a lone j obs a r e t o be execu t ed w i th no i n t e rdependenc i e s ,

t hen a spec i a l i z ed s chedu l e r may no t be r equ i r ed . Howeve r , i f you

wan t t o r e se rve a spec i f i c r e sou rce o r ensu re t ha t d i f f e r en t j obs

w i th in t he app l i c a t i on run concu r r en t l y ( fo r i n s t ance , i f t hey

r equ i r e i n t e r -p roce s s commun ica t i on ) , t hen a j ob s chedu l e r shou ld

be u sed t o coo rd ina t e t he execu t i on o f t he j obs . The G lobus

Too lk i t does no t i nc lude such a s chedu l e r , bu t t he r e a r e s eve ra l

s chedu l e r s ava i l ab l e t ha t have been t e s t ed w i th and can be u sed i n

a G lobus g r i d env i ronmen t . I t shou ld a l so be no t ed t ha t t he r e

cou ld be d i f f e r en t l eve l s o f s chedu l e r s w i th in a g r i d env i ronmen t .

Fo r i n s t ance , a c l u s t e r cou ld be r ep re sen t ed a s a s i ng l e r e sou rce .

The c lu s t e r may have i t s own schedu l e r t o he lp manage t he nodes

i t con t a in s . A h ighe r - l eve l s chedu l e r ( some t imes ca l l ed a me t a

s chedu l e r ) m igh t be u sed t o s chedu l e work t o be done on a

c l u s t e r , wh i l e t he c l u s t e r ' s s chedu l e r wou ld hand l e t he a c tua l

s chedu l i ng o f work on t he c lu s t e r ' s i nd iv idua l nodes .

F igu re 5 : Schedu l e r

9

6 .5 . Da ta managemen t

I f any da t a - - i nc lud ing app l i c a t i on modu le s - - mus t be

moved o r made acce s s ib l e t o t he nodes whe re an app l i c a t i on ' s

j obs w i l l execu t e , t hen t he r e needs t o be a s ecu re and r e l i ab l e

me thod fo r mov ing f i l e s and da t a t o va r i ous nodes w i th in t he

g r i d . The G lobus Too lk i t con t a in s a da t a managemen t componen t

t ha t p rov ide s such s e rv i ce s . Th i s componen t , know a s Gr id Acces s

t o Seconda ry S to r age (GASS) , i nc ludes f a c i l i t i e s such a s

Gr idFTP . Gr idFTP i s bu i l t on t op o f t he s t anda rd FTP p ro toco l ,

bu t adds add i t i ona l f unc t i ons and u t i l i z e s t he GSI fo r u se r

au then t i c a t i on and au tho r i za t i on . The re fo re , once a u se r ha s an

au then t i c a t ed p roxy ce r t i f i c a t e , he c an u se t he Gr idFTP f ac i l i t y t o

move f i l e s w i thou t hav ing t o go t h rough a l og in p roce s s t o eve ry

node i nvo lved . Th i s f a c i l i t y p rov ide s t h i rd -pa r t y f i l e t r ans f e r so

t ha t one node can i n i t i a t e a f i l e t r ans f e r be tween two o the r nodes .

F igu re 6 : Da t a managemen t

6 .6 . Job and r e source managemen t

With a l l t he o the r f a c i l i t i e s we have j u s t d i s cus sed i n p l ace ,

we now ge t t o t he co re s e t o f s e rv i ce s t ha t he lp pe r fo rm ac tua l

work i n a g r i d env i ronmen t . The Gr id Resou rce A l loca t i on

Manage r (GRAM) p rov ide s t he s e rv i ce s t o a c tua l l y l aunch a j ob

on a pa r t i cu l a r r e sou rce , check i t s s t a t u s , and r e t r i eve i t s r e su l t s

when i t i s comple t e .

10

F igu re 7 : Gram

6 .7 Job f l ow in a g r id env i ronmen t

Enab l i ng an app l i c a t i on fo r a g r i d env i ronmen t , i t i s

impor t an t t o keep i n m ind t he se componen t s and how they r e l a t e

and i n t e r ac t w i th one ano the r . Depend ing on you r g r i d

imp lemen ta t i on and app l i c a t i on r equ i r emen t s , t he r e a r e many

ways i n wh ich t he se p i ece s c an be pu t t oge the r t o c r ea t e a

so lu t i on .

7 . ADVANTAGES

Gr id compu t ing i s abou t ge t t i ng compu te r s t o work t oge the r .

A lmos t eve ry o rgan i za t i on i s s i t t i ng on t op o f eno rmous , unused

compu t ing capac i t y , w ide ly d i s t r i bu t ed . Ma in f r ames a r e i d l e 40%

o f t he t ime Wi th Gr id compu t ing , bus ine s se s c an op t im ize

compu t ing and da t a r e sou rce s , poo l t hem fo r l a rge capac i t y

work loads , sha r e t hem ac ros s ne tworks , and enab l e co l l abo ra t i on .

Many cons ide r Gr id compu t ing t he nex t l og i ca l s t ep i n t he

evo lu t i on o f t he In t e rne t , and ma tu r i ng s t anda rds and a d rop i n

t he cos t o f bandwid th a r e fue l i ng t he momen tum we ' r e

expe r i enc ing t oday . V i r t ua l i z a t i on o f t he compu t ing env i ronmen t .

8 . CHANLLANGES OF GRID

11

A word o f c au t i on shou ld be g iven t o t he ove r ly

en thus i a s t i c . The g r i d i s no t a s i l ve r bu l l e t t ha t c an t ake any

app l i c a t i on and run i t a 1000 t imes f a s t e r w i thou t t he need fo r

buy ing any more mach ine s o r so f twa re . No t eve ry app l i c a t i on i s

su i t ab l e o r enab l ed fo r r unn ing on a g r i d . Some k inds o f

app l i c a t i ons s imp ly canno t be pa r a l l e l i z ed . Fo r o the r s , i t c an t ake

a l a rge amoun t o f work t o mod i fy t hem to ach i eve f a s t e r

t h roughpu t . The con f igu ra t i on o f a g r i d c an g r ea t l y a f f ec t t he

pe r fo rmance , r e l i ab i l i t y , and s ecu r i t y o f an o rgan i za t i on ' s

compu t ing i n f r a s t ruc tu r e . Fo r a l l o f t he se r e a sons , i t i s impor t an t

f o r u s t o unde r s t and how f a r t he g r i d ha s evo lved t oday and wh ich

f ea tu r e s a r e coming t omor row o r i n t he d i s t an t f u tu r e

9 . CONCLUSION

G r id compu t ing i n t roduces a new concep t t o IT

i n f r a s t ruc tu r e s because i t suppo r t s d i s t r i bu t ed compu t ing ove r a

ne twork o f he t e rogeneous r e sou rce s and i s enab l ed by open

s t anda rds . Gr id compu t ing works t o op t im ize unde ru t i l i z ed

r e sou rce s , dec r ea se cap i t a l expend i t u r e s , and r educe t he t o t a l co s t

o f owne r sh ip . Th i s so lu t i on ex t ends beyond da t a p roce s s ing and

i n to i n fo rma t ion managemen t a s we l l . I n fo rma t ion i n t h i s con t ex t

cove r s da t a i n da t abase s , f i l e s , and s t o r age dev i ce s . I n t h i s

a r t i c l e , we ou t l i ne po t en t i a l p rob l ems and t he means o f so lv ing

t hem in a d i s t r i bu t ed env i ronmen t . .

12