+ All Categories
Home > Documents > 15-719-S17 L3 building a carnegie mellon cloud...

15-719-S17 L3 building a carnegie mellon cloud...

Date post: 02-Jun-2020
Category:
Upload: others
View: 12 times
Download: 3 times
Share this document with a friend
64
4CTWOTYR L 5L@YPRTP >PWWZY NWZCO 9L@BS 9TMAZY 9@PR 9LYRP@ >LUO DLV@ -/..-M0 3OaLYNPO NWZCO NZX>CBTYR D>@TYR (&-
Transcript
Page 1: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

4 TWOTYR L5L YPRTP >PWWZY NWZ O

9L S 9TM ZY 9 PR 9LYRP >LUO DLV

- / . . -M0 3OaLYNPO NWZ O NZX TYRD TYR (& -

Page 2: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>Fg OTWPXXL

j  Z ZQ NZX TYR P Z NP

j  5W P OP V Z OL LNPY P P N

j  bYPO Md OP L XPY Z ZUPN

j  A P d X NS PO ZYWd O TYR OPLOWTYP

j  :Zb Z LVP LOaLY LRP ZQ L P NdNWP 2

(

Page 3: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

>ZYP TeTYR L P NdNWP

j  TZY 0 CPY Z PY T P XLNSTYP

j  8Z P ZUPN bTWWTYR Z Ld QZ bSL PaP T TYLaLTWLMWP

j  TVP PY TYR L NLj  TZY (0 CPY Z Z TZY ZQ XLNSTYP

j  AZ TZY P LW Z NL LMTWT d bLY PO Md L P

j  TVP PY TYR L T P TY L SZ PW

j  3 TXP LNP SL TYR L ZLNS

)

Page 4: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

ESP TYT TLW 5>F NWZ O

j  F P PY Z TZY ZQ XLNSTYP

j  5>F RTaP Z P XLNSTYP LNNZ Y

j  F P P PY PO Z TZY L OP T PO

j  6PaPWZ P aTNP

j  6PaPWZ bPM T P

j  C Y LYLWd TN

Page 5: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O0 TYT TLW aP TZY

D Z LRP

>PXZ d

5ZX P

DSL PO Z LRP

F P P P Z PY Z TZY

F P )

F P (

A ZaT TZYP NSPO WP

F P

Page 6: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

A ZaT TZYP TY

j  4L TN0 I NZX P XPXZ d K Z LRP

j  >LdMP YP bZ V ZZ

j  3HD N PL P MTY ZQ SP P P R X WL RP

j  TVP hML TNi hOPW cPi h P TOPY TLWi T P

j  TZYLW L WPL Z TYT TLW Pj  DSL PO Z LRP YPPOPO2

j  6ZP OL L YPPO Z MP P T PY 2

,

Page 7: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

A ZaT TZYP Z

j  3 TRYXPY ZQ P Z XLNSTYP

j  CP T P XL NSTYR P P PO P Z NP Z XLNSTYP SL SLaP SPXLaLTWLMWP

j  6ZTYR Z PQQTNTPY Wd T L MTY LNVTYR ZMWPX

j  3 X TZY PO Z PO NP NZX WPcT d

j  >TRS TYaZWaP XTR L TYR PcT TYR P

j  6 P Z YPNP T d Z TYN PL PO PQQTNTPYNd

-

Page 8: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

ESP NSPO WP

j  5SZZ P bSTNS P g ZNP P Z Y

j  3WWZb NWZ O Z TZ T TeP NWTPY

j  3WWZb NWZ O Z ZaP M N TMP XLNSTYP

j  ?Z YPPO Z Y TY LYNP b YZ LN TaT d

j  5SZTNP OP PYO ZY P g ZNP P

j  7 R OT TM PO ZQ bL P SL Y TY WZNV P SZ WO MP NZNSPO WPO

.

Page 9: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O b NSPO WP

/

D Z LRP

>PXZ d

5ZX P

DSL PO Z LRP

F P )

F P (

F P 0 I KF P (0 3 4 5F P )0 6 7 8

F P P P Z PY Z TZY

A ZaT TZYP NSPO WP

F P

Page 10: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

7YNL WL TZY QZ NZX P Z LRP YP bZ VTYR LYO OL L

j  EZ T ZWL P NZ WZNL PO P ZNP P

j  F P PWPN PO ZQ bL P D PYaT ZYXPYj  3aZTO TY P QP PYNP O P Z PYaT NZYQTR NSLYRPj  3aZTO P QZ XLYNP TY P QP PYNPj  PP QTWP OL L TaL P

j  7Y P SL P ZM LTY P Z NP SPd LTO QZ

&

Page 11: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

YP Z TZY0 aT LWTeL TZY

j  > W T WPcP TYRWP P Z NP LXZYR X W T WP ZQ bL P ML PO P Z NP

j  :L ObL P D 1 aT LW XLNSTYP

j  ?P bZ V 1 G 3?

j  6T V 1 aT LW OT V

j  F PO Md0 3XLeZY CLNV LNP P N

6ZP YZ ZWaP LWW ZMWPX P R P QZ XLYNP TY P QP PYNP

Page 12: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O b PYNL WL TZY

(

D Z LRP

>PXZ d

5ZX P

DSL PO Z LRP

F P )

F P P P Z PY Z TZY

F P (

F P 0 I KF P (0 3 4 5F P )0 6 7 8

A ZaT TZYP NSPO WP

F P

Page 13: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

ESP YPPO QZ QL W ZWP LYNP

j  :L ObL P ZQ bL P XLd QLTW L LYd TXP

j  XTWWTZY SL O OT V bT S XTWWTZY SZ >E48 k1 SL O OT V QLTW P SZ

j  6 TYR QT dPL ZQ L 9ZZRWP NW P g WTQP0

j  &&& XLNSTYP QLTW P ) Z P QLTW P k&&& ZQ OT V QLTW P 6PLY 9ZZRWP

j  ?L LW OT L P bT P Z OL LNPY P

j  5ZY N P TWTPY Z QLTW P P aTNP

)

Page 14: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

3 QPb QL W ZWP LYNP XPNSLYT X

j  8Z Z LRP0

j  C3 6 P WTNL TZY bT STY LN Z OL LNPY P

j  8Z N T TNLW TYQ L N P LYO P aTNP

j  D L P XLNSTYP P WTNL TZY NSPNV ZTY TYR

j  F P L P Q PP ZQ bL P OP TRY

Page 15: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>Fg f LLDg NWZ O

5>FAT M RS

5>FBL L

9PZ P WTNL TZY

CP WTNL TZY

Page 16: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

SP YTaP T TP UZTY SP Q Ld

j  4P VPWPd TY ZO NP LY NWZ O ZQQP TYR

j  DTXTWL Z 5>Fg PcT TYR NWZ O

j  5>F YPPO Z LOO aLW P Z SPT ZQQP TYR

j  ESTYV MLNV Z NL P OTP

,

Page 17: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5L P OTP OP T PO QPL P

j  DNLWLMWP QL W ZWP LY M TWOTYR MWZNV

j  7 R VPd aLW P Z P QTWP d PX P N

j  A ZR LXXTYR XZOPW Q LXPbZ V

j  8Z LYLWd TN M TWOTYR L bPM P aP P N

j  3 ZXL PO XPNSLYT X QZ PWL TN NLWTYR

j  EZ SLYOWP aL dTYR OPXLYO

-

Page 18: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O b QPL P M TWOTYR MWZNV LM LN TZY

.

4 TWOTYR MWZNV

6dYLXZ64

DTX WP64

FLR

PX

ZYT Z

DSL PO Z LRP

A ZaT TZYP NSPO WP

Page 19: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

A ZR LXXTYR Q LXPbZ V

j  :TRS WPaPW Q LXPbZ V LYO WLYR LRP QZ PQQTNTPY Wd ZNP TYR OTQQP PYOL L d P

j  7 R >L CPO NP 6 dLO D L V

j  >Z L P NWZ O TY LYO ZQ SPX PWaP

j  :LaP SPT ZbY NSPO WP

j  :LaP SPT ZbY QL W ZWP LYNP XPNSLYT X

/

Page 20: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O bZR LXXTYR Q LXPbZ V

(&

4 TWOTYR MWZNVA ZR LXXTYRQ LXPbZ V

EbZ WPaPW NSPO WTYR

FLR

PXZY

T Z

DSL PO Z LRP

A ZaT TZYP NSPO WP

Page 21: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

7WL TN NLWTYR

j  M P aPO WZLO QZ L P aTNP T aL TLMWP

j  :TRS WZLO QZ I4Zc LWP O TYR SLYV RTaTYR

j  E LOT TZYLW ZW TZY0 ZaT TZY QZ PLV

j  ESP 7WL TN NLWTYR L ZLNS0

j  5WZ O XZYT Z WZLO

j  3OO L WTNL TZY TY LYNP L YPNP L d

(

Page 22: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O bPWL TN NLWTYR

((

4 TWOTYR MWZNVA ZR LXXTYRQ LXPbZ V

7WL TN NLWP

FLR

PXZY

T Z

DSL PO Z LRP

A ZaT TZYP NSPO WP

Page 23: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

>ZYT Z TYR OTLRYZ T

j  5WZ O XZYT Z P L WTNL TZY

j  A ZaTOP LWP bSPY S P SZWO N Z PO

j  7 R 9LYRWTL 3HD NWZ O bL NS

j  5WZ O ZaTOP LNTYR WTM L TP

j  3YLWdeP LNP QZ ZMWPX ZZ NL P

j  7 R 6L P

()

Page 24: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

5>F NWZ O bXZYT Z TYR OTLRYZ T

(

4 TWOTYR MWZNVA ZR LXXTYRQ LXPbZ V

>ZY

TZTYR

OTLR

YZT

7WL TN NLWP

FLR

PXZY

T Z

DSL PO Z LRP

A ZaT TZYP NSPO WP

Page 25: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

FLR

PXZY

T Z5>Fg NWZ O

(

FLR

PXZY

T Z

4 TWOTYR MWZNVA ZR LXXTYRQ LXPbZ V

>ZY

TZTYR

OTLR

YZT

7WL TN NLWP

DSL PO Z LRP

A ZaT TZYP NSPO WP

Page 26: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

1 - /0 3OaLYNPO NWZ O NZX TYR 5>FCPaT TZY0 & ( (& -

?Pc NWL LYO ZYbL O

j  A ZR LXXTYR Q LXPbZ V

j  L Pj  7YNL WL TZY ZQ NZX L TZY G>

j  DNSPO WTYR

j  D Z LRP

j  7 N

(,

Page 27: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Open source virtual infrastructure (VI) for private clouds 15-719/18-847b Greg Ganger Garth Gibson Majd Sakr

Jan 25, 2016 15719/18847b Adv. Cloud Computing 1

Page 28: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenNebula Architecture pic from [Sotomayor2009]

Jan 23, 2016 15719/18847b Adv. Cloud Computing 2

Page 29: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Example open source VI mgmt software

•  OpenNebula (for deploying and managing groups of 1-or-more VMs) o  Automates aspects of VM setup on physical hosts

•  e.g., preparing disk images, setting up networking, etc.

o  Works for several VM types (e.g., KVM and Vmware) and external clouds •  Note: external cloud support enables hybrid cloud deployments

•  Haizea (for managing “leasing” of resources) o  Can be used as a scheduling component of OpenNebula

•  Decides which hardware resources are assigned to a group of VMs (and when) •  Claims novelty around reservation and preemption/migration support

Jan 23, 2016 15719/18847b Adv. Cloud Computing 3

Page 30: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenNebula Architecture pic from [Sotomayor2009]

Jan 23, 2016 15719/18847b Adv. Cloud Computing 4

Page 31: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenNebula architecture components

•  OpenNebula core (orchestrates use of the others)

•  Virtualization drivers o  VM-type-specific functions for setting up VMs, starting/stopping VMs, etc.

•  Network drivers o  Functions for assigning network addresses

•  Storage drivers o  Functions for attaching network storage resources

•  External cloud drivers o  Functions for putting some VMs on an external cloud (e.g., EC2)

•  Scheduler: decides which VMs get which physical resources

Jan 23, 2016 15719/18847b Adv. Cloud Computing 5

Page 32: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

15#719/18#847b*Advanced*Cloud*Compu8ng*

Lecture*03**

OpenStack**

January*25,*2017**

hGp://www.cs.cmu.edu/~15719/**

Page 33: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Mo8va8on*

•  Users*locate*compu8ng*resources*based*on:*– HW*architecture*– Memory*capacity*–  Storage*capacity*– Network*connec8vity*– Geo*loca8on*

•  Involves:*–  Resource*availability*– Applica8on*and*performance*profiling*–  SoXware*service*requirements*

Page 34: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Resource*Provisioning*

•  Resource*provisioning*for*large*#*of*resources*– Contact*several*providers*– Get*heterogeneous*resources*

•  Performance*profiling*becomes*difficult*

•  Efficient*use*of*resources*becomes*difficult*

•  Few*users*can*exploit*heterogeneity*– Uniformity*makes*applica8on*development*and*deployment*easier*

Page 35: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Goal*of*the*Cloud*IAAS*Model*

•  Convert*a*manual*large#scale*resource*provisioning*and*programming*problem*into*elas8c*u8lity*(cloud*compu8ng)*–  Self*deployment*model*

•  Current*public*cloud*IAAS*offerings*are*proprietary*– Do*not*allow*

•  Experimenta8on*•  Instrumenta8on*•  Deployment*of*a*private*cloud*

•  Private*cloud*soXware*systems*allow*users*to*customize,*extend,*and*experiment*with*management*infrastructure*

Page 36: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

IAAS*

•  To*offer*cloud*compu8ng*services*running*on*standard*hardware.*–  SoXware*defined*datacenter*as*a*service*

•  Deploy*a*private*cloud*•  Offer*an*IAAS*public*cloud*service*•  Instrument*to*answer*open*ques8ons:*

1.  What*is*the*right*distributed*architecture*for*a*cloud*compu8ng*system?*

2.  What*resource*characteris8cs*must*VM*schedulers*consider*to*make*the*most*efficient*use*of*the*resources?*

Page 37: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Cloud*Interfaces*&*Abstrac8ons*

•  Cloud*offers*a*spectrum*–  From*IaaS*

•  Dynamically*provision*VMs,*Storage,*networking*

–  To*SaaS*•  Flexible*access*to*hosted*services*

•  All*resources*–  Should*be*well*defined*–  Provide*reasonably*determinis8c*performance*–  Can*be*allocated*and*de#allocated*on*demand*–  Should*provide*some*level*of*security*guarantees*through*isola8on*

Page 38: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Cloud*Objec8ves*and*Design*Goals*

•  Rapidly*scale*up*and*down*as*load*fluctuates*•  Support*a*large*number*of*users*requiring*resources*on*demand*

•  Provide*stable*access*to*provided*resources*over*the*Internet*

•  Op8mize*use*of*capital*compu8ng*infrastructure*– Avoid*having*par8ally*idle*computers*when*they*could*be*doing*something*useful*instead*

Page 39: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Cloud*Service*Configura)on*Management*

Health/Status*Monitoring*

Cloud*Service*

User*

Cloud*Users*and*Services*

Page 40: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Interfaces*

•  Applica8on*development/deployment*user*– Sign*up,*creden8aling,*query*system*

– Create,*modify,*interrogate*data*

– Provision,*submit*job*

•  Administrator*– Manage*user*accounts,*inspect*availability*

– Monitor*and*manage*resources*

Page 41: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Services*Needed*for*IAAS?*

IAAS*

Compute*

Networking*

Dashboard*

Storage*

OS*Images*

Orchestra#*8on*

Iden8ty*

Telemetry*

Page 42: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*–*1*•  Controls*pools*of*compute,*storage,*and*networking*resources*in*a*datacenter.*

•  Managed*through*a*dashboard*that*gives*–  administrators*control*

–  users*ability*to*provision*resources*through*a*web*interface.*

hGp://ilearnstack.com/2013/04/23/introduc8on#to#openstack#2/*

Page 43: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*–*2*

•  Joint*effort*by*Rackspace*and*NASA*in*2010*•  Community,*OpenStack*Founda8on,*defines*official*OpenStack*components*

•  Gaining*momentum*– Open,*no*trade*secrets*– Companies*are*contribu8ng*to*open*source*

– Can*be*integrated*with*proprietary*components*

•  Release*cycle*is*6#months*

Page 44: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*–*3*•  Open*source*cloud*compu8ng*framework*that*uses*compu8ng*and*storage*infrastructure*to*provide*a*plamorm*to*offer*cloud*services*on*standard*hardware.*– Offer*a*common*open#source*framework*and*a*community*will*form*•  As*of*2016,*500*companies*have*joined*

– Offers*compa8bility*with*Amazon*Web*Services*•  Such*as*EC2,*S3,*EBS,*…*

•  Provides*management*infrastructure*for*exis8ng*underlying*technologies*–  E.g.,*manages*usual*QEMU/KVM*or*Xen*hypervisor,**Linux*bridge*and*VXLAN*networking,*etc.*

Page 45: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*–*4*

•  OpenStack*consists*of*several*independent*parts,*named*the*OpenStack*services.*

•  All*services*authen8cate*through*a*common*Iden8ty*service.*

•  Individual*services*interact*with*each*other*through*public*APIs,*except*where*privileged*administrator*commands*are*necessary.*

•  Provides*a*modular*architecture*to*reuse*exis8ng*infrastructure*to*manage*various*types*of*resources*(e.g.,*virtual*machine,*container,*“bare*metal”*physical*machine)*

Page 46: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Independent*Services*

Public*APIs*

Compute*

Networking*

Dashboard*

Storage*

OS*Images*

Orchestra#*8on*

Iden8ty*

Telemetry*

Page 47: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*Services*Service* Codename*

•  OpenStack*Compute* •  Nova*•  OpenStack*Object*Storage* •  SwiX*•  OpenStack*Block*Storage* •  Cinder*•  OpenStack*Networking* •  Neutron*•  OpenStack*Dashboard* •  Horizon*•  OpenStack*Iden8ty* •  Keystone*•  OpenStack*Image*service* •  Glance*•  OpenStack*Telemetry*service* •  Ceilometer*

•  OpenStack*Orchestra8on*service* •  Heat*•  OpenStack*Database*service* •  Trove*•  OpenStack*Data*processing*service* •  Sahara*

Page 48: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Conceptual*Architecture*

Service* Codename*Compute* Nova*

Object*Storage* SwiX*

Block*Storage* Cinder*

Networking* Neutron*

Dashboard* Horizon*

Iden8ty* Keystone*

Image*service* Glance*

Telemetry*service* Ceilometer*

Orchestra8on*service* Heat*

Database*service* Trove*

Data*processing*service* Sahara*

hGp://docs.openstack.org/admin#guide#cloud/common/get_started_conceptual_architecture.html*

Page 49: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Access,*Communica8on*and*APIs*•  All*OpenStack*services*have*at*least*one*API*process*–  Listens*for*API*requests,*preprocesses*them*and*passes*them*on*to*other*parts*of*the*service.*

•  For*communica8on*between*the*processes*of*one*service*–  An*advanced*message*queuing*protocol*(AMQP)*message*broker*is*used.*

•  Users*can*access*OpenStack*via*–  the*web#based*user*interface*implemented*by*the*dashboard*–  command#line*clients**–  issuing*API*requests*through*tools,*browser*plug#ins*or*curl*

•  For*applica8ons,*several*SDKs*are*available.*•  All*access*methods*issue*REST*API*calls*to*the*various*OpenStack*services.*

Page 50: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

hGps://ilearnstack.files.wordpress.com/2013/04/openstack#arch#grizzly#v1#logical.jpg*

Dashboard*

Compute*Service*

Block*Storage*

Object*Store* Network*

Service*Image*Service*

Iden8ty*Service*

Page 51: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

hGps://ilearnstack.files.wordpress.com/2013/04/openstack#arch#grizzly#v1#logical.jpg*

hGp://docs.openstack.org/ops#guide/_images/osog_0001.png**

Page 52: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Logical*Architecture*

hGp://docs.openstack.org/admin#guide#cloud/common/get_started_logical_architecture.html*

Page 53: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups
Page 54: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups
Page 55: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Keystone*–*OpenStack*Iden8ty*

•  Provides*authen8ca8on*and*authoriza8on*for*other*OpenStack*services*and*users*

•  Provides*a*catalog*of*endpoints*for*all*OpenStack*services*(service*discovery)*

•  Central*to*most*OpenStack*opera8ons,*hence*the*name* (openstack) endpoint list --service identity -c "Service Name" -c "Service Type" -c "Interface" -c "URL" +--------------+--------------+-----------+------------------------------------------------+ | Service Name | Service Type | Interface | URL | +--------------+--------------+-----------+------------------------------------------------+ | keystone | identity | admin | http://kiska.pdl.local.cmu.edu:35357/v3 | | keystone | identity | internal | http://kiska-control.pdl.local.cmu.edu:5000/v3 | | keystone | identity | public | http://kiska.pdl.local.cmu.edu:5000/v3 | +--------------+--------------+-----------+------------------------------------------------+ (openstack) endpoint list --service nova -c "Service Name" -c "Service Type" -c "Interface" -c "URL" +--------------+--------------+-----------+----------------------------------------------------------------+ | Service Name | Service Type | Interface | URL | +--------------+--------------+-----------+----------------------------------------------------------------+ | nova | compute | internal | http://kiska-control.pdl.local.cmu.edu:8774/v2.1/%(tenant_id)s | | nova | compute | admin | http://kiska.pdl.local.cmu.edu:8774/v2.1/%(tenant_id)s | | nova | compute | public | http://kiska.pdl.local.cmu.edu:8774/v2.1/%(tenant_id)s | +--------------+--------------+-----------+----------------------------------------------------------------+

Courtesy*of*Chad*Dougherty*

Page 56: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Nova*–*OpenStack*Compute*•  Use*nova*to*host*and*manage*cloud*compu8ng*systems.*

•  Nova*is*a*major*part*of*an*Infrastructure#as#a#Service*(IaaS)*system.*Main*modules*are*implemented*in*Python*(as*are*most*components).*

•  OpenStack*Compute,*nova,*interacts*with*– OpenStack*Iden8ty*for*authen8ca8on;*– OpenStack*Image*service*for*disk*and*server*images;*and**– OpenStack*dashboard*for*the*user*and*administra8ve*interface.**

•  OpenStack*Compute*can*scale*horizontally*on*standard*hardware,*and*download*images*to*launch*instances.*

Page 57: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Nova*–*API*•  nova-api*service*

–  Accepts*and*responds*to*end*user*compute*API*calls.*The*service*supports*the*OpenStack*Compute*API,*the*Amazon*EC2*API,*and*a*special*Admin*API*for*privileged*users*to*perform*administra8ve*ac8ons.*It*enforces*some*policies*and*ini8ates*most*orchestra8on*ac8vi8es,*such*as*running*an*instance.*

•  nova-compute*service*–  A*worker*daemon*that*creates*and*terminates*virtual*machine*instances*

through*hypervisor*APIs.*For*example:*•  XenAPI*for*XenServer/XCP*•  libvirt*for*KVM*or*QEMU*•  VMware*API*for*VMware*

–  Processing*is*fairly*complex.*Basically,*the*daemon*accepts*ac8ons*from*the*queue*and*performs*a*series*of*system*commands*such*as*launching*a*KVM*instance*and*upda8ng*its*state*in*the*database.*

•  nova-scheduler*service*–  Takes*a*virtual*machine*instance*request*from*the*queue*and*determines*

on*which*compute*server*host*it*runs.*

hGp://docs.openstack.org/admin#guide#cloud/common/get_started_compute.html*

Page 58: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Nova*–*API*•  nova*client*

–  Enables*users*to*submit*commands*as*a*tenant*administrator*or*end*user.*

•  The*queue*–  A*central*hub*for*passing*messages*between*daemons.*Usually*implemented*with*RabbitMQ,*but*can*be*implemented*with*an*AMQP*message*queue,*such*as*Apache*Qpid*or*Zero*MQ.*

•  SQL*database*–  Stores*most*build#8me*and*run#8me*states*for*a*cloud*infrastructure,*including:*•  Available*instance*types*

–  Instances*in*use*–  Available*networks*–  Projects*

–  Theore8cally,*OpenStack*Compute*can*support*any*database*that*SQL#Alchemy*supports.*Common*databases*are*SQLite3*for*test*and*development*work,*MySQL,*and*PostgreSQL.*

Page 59: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Ceilometer*–*OpenStack*Telemetry*•  Performs*the*following*func8ons:*– Polls*metering*data*related*to*OpenStack*services.*

– Collects*event*and*metering*data*by*monitoring*no8fica8ons*sent*from*services.*

– Publishes*collected*data*to*various*targets*including*data*stores*and*message*queues.*

– Creates*alarms*when*collected*data*breaks*defined*rules.*

Page 60: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Ceilometer*–*API*•  ceilometer-agent-compute*#*A*compute*agent*

–  Runs*on*each*compute*node*and*polls*for*resource*u8liza8on*sta8s8cs.**

•  ceilometer-agent-central**#*A*central*agent*–  Runs*on*a*central*management*server*to*poll*for*resource*u8liza8on*sta8s8cs*for*resources*not*

8ed*to*instances*or*compute*nodes.*Mul8ple*agents*can*be*started*to*scale*service*horizontally.*

•  ceilometer-agent-notification**#*A*no8fica8on*agent*–  Runs*on*a*central*management*server(s)*and*consumes*messages*from*the*message*queue(s)*to*

build*event*and*metering*data.*

•  ceilometer-collector**#*A*collector*–  Runs*on*central*management*server(s)*and*dispatches*collected*telemetry*data*to*a*data*store*or*

external*consumer*without*modifica8on.*

•  ceilometer-alarm-evaluator*#*An*alarm*evaluator*–  Runs*on*one*or*more*central*management*servers*to*determine*when*alarms*fire*due*to*the*

associated*sta8s8c*trend*crossing*a*threshold*over*a*sliding*8me*window.*

•  ceilometer-alarm-notifier*#*An*alarm*no8fier*–  Runs*on*one*or*more*central*management*servers*to*allow*alarms*to*be*set*based*on*the*threshold*

evalua8on*for*a*collec8on*of*samples.*

•  ceilometer-api*#*An*API*server*–  Runs*on*one*or*more*central*management*servers*to*provide*data*access*from*the*data*store.*

hGp://docs.openstack.org/admin#guide#cloud/common/get_started_telemetry.html*

Page 61: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Storage*Concepts*

On;instance*/*ephemeral* Block*storage*(cinder)* Object*Storage*(swiD)*

•  Runs*opera8ng*systems*and*provides*scratch*space*

•  Used*for*adding*addi8onal*persistent*storage*to*a*virtual*machine*(VM)*

•  Used*for*storing*virtual*machine*images*and*data*

•  Persists*un8l*VM*is*terminated* •  Persists*un8l*deleted* •  Persists*un8l*deleted*

•  Access*associated*with*a*VM* •  Access*associated*with*a*VM* •  Available*from*anywhere*

•  Implemented*as*a*filesystem*underlying*OpenStack*Compute*

•  Mounted*via*OpenStack*Block*Storage*controlled*protocol*(for*example,*NFS*or*iSCSI)*

•  REST*API*

•  Encryp8on*is*available* •  Encryp8on*is*available*•  Easily*scalable*for*future*growth*

•  Administrator*configures*size*sexng,*based*on*flavors* •  Sizings*based*on*need*

•  Example:*10*GB*first*disk,*30*GB/core*second*disk* •  Example:*1*TB*“extra*hard*drive”*

•  Example:*10s*of*TBs*of*data*set*storage*

hGp://docs.openstack.org/admin#guide#cloud/common/get_started_storage_concepts.html*

Page 62: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Horizon*–*OpenStack*Dashboard*•  A*modular*Django*web*applica8on*that*provides*a*graphical*

interface*to*OpenStack*services.*

•  Not*a*necessary*component*to*deploy*OpenStack.*

*

Page 63: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

OpenStack*and*AWS*

OpenStack* AWS*

Object*Storage* SwiX* S3*

Block*Storage* Cinder* Elas8c*Block*Storage*(EBS)*

Compute* Nova* EC2*

Telemetry* Ceilometer* CloudWatch*

Data*Processing* Sahara* EMR*

Page 64: 15-719-S17 L3 building a carnegie mellon cloud garthgarth/15719/lectures/15719-S17-L3-cloud...Example open source VI mgmt software • OpenNebula (for deploying and managing groups

Next*Time*•  Monday,*1/30/2017:*– Programming*Frameworks/models*

– P2.1*is*out*•  Overview*


Recommended