69
What’s Up Next for Heat? Alex Heneveld / @ahtweetin OpenStack Israel 27 May 2013
| Date post: | 15-Jan-2015 |
| Category: |
Technology |
| Upload: | alexheneveld |
| View: | 1,043 times |
| Download: | 0 times |
What’s Up Next for Heat?
Alex Heneveld / @ahtweetinOpenStack Israel27 May 2013
CloudIt’s not just about VM’s
It’s about storage, networking, maas, ...
It’s not just about VM’s
Cloud
Ultimately it’s about applications
CloudIt’s not just about VM’s
It makes the clouds rise
Declarative — OpenStack Heat (Grizzly) and AWS CloudFormationProcedural — Apache Whirr, Gigaspaces Cloudify, Cloudsoft BrooklynOpenStack Summit — Red Hat, RAX, IBM, HP, Cloudsoft, OASIS TOSCA and CAMPDSLs in Progress — CAMP PDP and OpenStack Heat (Havana)
Approaches to Applications in the Cloud
AWS CloudFormation
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "WordPress is web software …", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair for SSH access", "Type" : "String" }, ... }, "Mappings" : { "AWSInstanceType2Arch" : { "m1.small" : { "Arch" : "64" }, "m1.large" : { "Arch" : "64" }, ... }, "AWSRegionArch2AMI" : { "us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } }, "Resources" : { "WebServer": { "Type": "AWS::EC2::Instance", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc-c++" : [], "make" : [],
...
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "WordPress is web software …", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair for SSH access", "Type" : "String" }, ... }, "Mappings" : { "AWSInstanceType2Arch" : { "m1.small" : { "Arch" : "64" }, "m1.large" : { "Arch" : "64" }, ... }, "AWSRegionArch2AMI" : { "us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } }, "Resources" : { "WebServer": { "Type": "AWS::EC2::Instance", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc-c++" : [], "make" : [],
...
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
AWS CloudFormation
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
Parameters
Mappings(config metadata)
WaitConditions &WaitHandles
(orchestration)
Outputs
Functions
AWS CloudFormation
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
AWS CloudFormation
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
AWS CloudFormation
"Type": "AWS::AutoScaling::AutoScalingGroup"
"Type": "AWS::EC2::EIP"
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
"Type": "AWS::SQS::Queue"
"Type": "AWS::CloudFormation::Stack"
"Type": "AWS::S3::Bucket"
AWS CloudFormation
"Type": "AWS::AutoScaling::AutoScalingGroup"
"Type": "AWS::EC2::EIP"
"Resources" : { "WebServer": { "Type": "AWS::EC2::Instance"
"Type": "AWS::SQS::Queue"
"Type": "AWS::CloudFormation::Stack"
"Type": "AWS::S3::Bucket"
AWS CloudFormation
Composition!
— Limited to AWS cloud
— Limited to AWS concepts
— Cumbersome to write
— Limited dependency injection
Nice, but:
AWS CloudFormation
— CloudFormation clone
— Targetting OpenStack resources and AWS compatibility
— Adds YAML support (comments and easier)
— Multi-cloud support using Apache Deltacloud
OpenStack Heat
(AWS CloudFormation){ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "WordPress is web software …", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair for SSH access", "Type" : "String" }, ... }, "Mappings" : { "AWSInstanceType2Arch" : { "m1.small" : { "Arch" : "64" }, "m1.large" : { "Arch" : "64" }, ... }, "AWSRegionArch2AMI" : { "us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } }, "Resources" : { "WebServer": { "Type": "AWS::EC2::Instance", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc-c++" : [], "make" : [],
...
OpenStack Heat
OpenStack Heat
HeatTemplateFormatVersion: 2012-12-12 Description: WordPress is web software ... Parameters: KeyName: Description: Name of an existing EC2 KeyPair for SSH access Type: String ... Mappings: AWSInstanceType2Arch: m1.small: {Arch: '32'} m1.large: {Arch: '64'} ... DistroArch2AMI: F18: {'32': F18-i386-cfntools, '64': F18-x86_64-cfntools}, ... Resources: WebServer: Type: AWS::EC2::Instance Metadata: AWS::CloudFormation::Init: config: packages: yum: gcc-c++: [] make: []
...
— CloudFormation clone
— Targetting OpenStack resources and AWS compatibility
— Adds YAML support (comments and easier)
— Multi-cloud support using Apache Deltacloud
OpenStack Heat
Available NOW in Grizzly!
OpenStack Heat
— Still limited to selected clouds
— Still limited in concepts
— Only slightly less cumbersome to write
— Very limited dependency injection
But...
Procedural Approach
10 INPUT "What app do you like? ", A$20 INPUT "What cloud do you like? ", C$30 GOSUB 10040 END
100 REM TODO deploy C$ to A$110 RETURN
Procedural Approach
— Millions of lines of bash, chef, python, ruby, java, ... or ...
— Apache Whirr
— Gigaspaces Cloudify
— Brooklyn and Cloudsoft AMP
Apache Whirr
— Apache top level project
— Deploy multi-node applications— Hadoop, Hama, Mahout, web,
Cassandra & many more
— Define topologies declaratively— Add new roles, configuration,
constraints and wiring procedurally
provision
install
configure
manage
Apache Whirr
— Apache top level project
— Deploy multi-node applications— Hadoop, Hama, Mahout, web,
Cassandra & many more
— Define topologies declaratively— Add new roles, configuration,
constraints and wiring procedurally
$ whirr launch-cluster \ --cluster-name=hbase \ --instance-templates='1 hbase-master,6 hbase-regionserver'
Apache Whirr
— Apache top level project
— Deploy multi-node applications— Hadoop, Hama, Mahout, web,
Cassandra & many more
— Define topologies declaratively— Add new roles, configuration,
constraints and wiring procedurally
spec = new ClusterSpec();spec.setProvider("cloudservers-uk");spec.setIdentity(apikey);spec.setCredential(secret);spec.setClusterName("hbase");spec.setInstanceTemplates(ImmutableList.of( new InstanceTemplate(1, "hbase-master"), new InstanceTemplate(6, "hbase-regionserver")));
cluster = new ClusterController().launchCluster(spec);
— Limited parameterisation
— Defining new roles requires coding
— Strict phases limit applicability
But...
— Composable to one dimension
— Portable locations support
— Pluggable with bash, Chef, Puppet, and more
Apache Whirr
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
application { name="petclinic" service { name = "mysql" } service { name = "tomcat" dependsOn = ["mysql"] } }
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
service { name "mysql" icon "mysql.jpg" type "DATABASE"
lifecycle{ install "mysql_install.groovy" start "mysql_start.groovy" ... }}
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
def installLinuxMysql(config,context,builder,currVendor,installScript,myCnfObject) {" builder.sequential {" " chmod(dir:"${context.serviceDirectory}", perm:"+x", includes:"*.sh")" " exec(executable: "${context.serviceDirectory}/${installScript}",failonerror: "true")}...switch (currVendor) { case ["Ubuntu", "Debian", "Mint"]: context.attributes.thisInstance["binFolder"]="/usr/bin" installLinuxMysql(config,context,builder,currVendor,"installOnUbuntu.sh",myCnfObject) ...
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
if sudo grep -q -E '[^!]requiretty' /etc/sudoers; then echo "Defaults:`whoami` !requiretty" | sudo tee /etc/sudoers.d/`whoami` >/dev/null sudo chmod 0440 /etc/sudoers.d/`whoami`fisudo apt-get -y -q update || error_exit $? "Failed on: sudo apt-get -y update"killMySqlProcesssudo apt-get -y -q purge mysql-client* mysql-server* mysql-common*sudo rm -rf /etc/mysql || error_exit $? "Failed on: sudo rm -rf /etc/mysql"sudo DEBIAN_FRONTEND='noninteractive' apt-get -o Dpkg::Options::='--force-confnew' -q -y \ install mysql-server-core mysql-server mysql-client mysql-common || error_exit $? "Failed ... ”
Gigaspaces Cloudify
— Apache-licensed Open PaaS Stack— Deployment and Management
— Build applications declaratively— Define services procedurally— Configure auto-scaling declaratively
scalingRule { serviceStatistics { metric "Requests per second" movingTimeRangeInSeconds 20 }
highThreshold { value 100 instancesIncrease 1 } }
— Defining new services requires coding
But...
— Composable to one dimension
— Parameterisable recipes
— Portable locations support
— Pluggable with bash, Chef, Puppet, and more
Gigaspaces Cloudify
Brooklyn & Cloudsoft AMP
— Brooklyn = Apache-licensed open-source
— AMP = Enterprise + Support
— Deployment and Management of Anything
— Hierarchical Autonomic Policies
— Define blueprints procedurally
Brooklyn
Contention ManagementCustom PaaSBig Data
public class MyWebCluster extends AbstractApplication implements MyWebClusterConstants { public void init() { MySqlNode mysql = addChild(EntitySpecs.spec(MySqlNode.class) .configure("creationScriptUrl", DB_SETUP_SQL_URL)); ControlledDynamicWebAppCluster web = addChild(EntitySpecs.spec( ControlledDynamicWebAppCluster.class) .configure(WebAppService.HTTP_PORT, PortRanges.fromString("8080+")) .configure(JavaWebAppService.ROOT_WAR, WAR_PATH) .configure(javaSysProp("brooklyn.example.db.url"), formatString("jdbc:%s%s?user=%s\\&password=%s", attributeWhenReady(mysql, MySqlNode.MYSQL_URL), DB_TABLE, DB_USERNAME, DB_PASSWORD)). .configure(JcloudsLocationConfig.MIN_RAM, 2048) .configure(JcloudsLocationConfig.TEMPLATE_SPEC, "osFamily=centos,os64Bit=true") ); }
}
Brooklyn: Nested Elastic Blueprints
public class MyWebCluster extends AbstractApplication implements MyWebClusterConstants { public void init() { MySqlNode mysql = ... ControlledDynamicWebAppCluster web = ... web.addEnricher(HttpLatencyDetector.builder(). url(ROOT_URL). rollup(10, SECONDS).build());
web.getCluster().addPolicy(AutoScalerPolicy.builder(). metric(REQUESTS_PER_SECOND_IN_WINDOW_PER_NODE). metricRange(10, 100).sizeRange(2, 5).build());
addEnricher(SensorPropagatingEnricher.newInstanceListeningTo(web, ROOT_URL, REQUESTS_PER_SECOND_IN_WINDOW, REQUEST_LATENCY_IN_SECONDS_IN_WINDOW)); }
}
Brooklyn: Policies, Enrichers and KPI’s
Brooklyn
— Defining blueprints requires coding
But...
— Composable and substitutable
— Parameterisable blueprints
— Portable and powerful locations support
— Pluggable with VM images, Heat, Chef, more
Applications
— Components (IaaS-resources)
— DeclarativeInfrastructure
— Procedural
— Extensible (new resources)
— Portable (hybrid, metal)
— Management
— Composable (stacks)
Heat in Grizzly
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Exte
nsib
le
Port
able
thanks largely to Red Hat
Portland Design Summit — Heat
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Port
able
Com
pone
nts
Exte
nsib
le
Portland Design Summit — Heat
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Port
able
Com
pone
nts
Exte
nsib
lelots of new interest and ideas
Portland Design Summit — Heat
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Man
agem
ent
Port
able
Ease
-of-
Use
Exte
nsib
leingredients from Brooklyn, Whirr, OoO and others
Portland Design Summit — Heat
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Man
agem
ent
Port
able
Exte
nsib
le
Auto
-wiri
nga special ingredient from Rackspace
Ease
-of-
Use
Portland Design Summit — Heat
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Man
agem
ent
Exte
nsib
le
Auto
-wiri
ng
Rela
tions
hips
Ease
-of-
Use
Port
able
a special ingredient from IBM
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Auto
-wiri
ng
Rela
tions
hips
Exte
nsib
le
Ease
-of-
Use
Port
able
Portland Design Summit — Heat
Heat in Havana
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Auto
-wiri
ng
Rela
tions
hips
Exte
nsib
le
Ease
-of-
Use
Port
able
ConcurrentDistributedResilientFunctions / DI
Heat in Havana
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Auto
-wiri
ng
Rela
tions
hips
Exte
nsib
le
Ease
-of-
Use
Port
able
ConcurrentDistributedResilientFunctions / DIStandards ?
OasisTOSCA
http://www.oasis-open.org/committees/tosca/
Topology and Orchestration Specification for Cloud Applications
Modeling Topologies with TOSCA Service Topologies are described using the TOSCA “Meta-model”:
Artifacts Describe Installables and Executables required to
instantiate and manage a service. Currently, they include:
Implementation Artifacts: – Executables or Plans that implement a Node’s
or Relationship’s Operations (e.g. a Bash script) Deployment Artifacts:
– Installables of the components (e.g. a TAR file)
A service’s Topology Model is included in a TOSCA Service Template which is packaged and shared, along with all dependent artifacts, as a TOSCA Cloud Service Archive (CSAR)
Service Templates Group the nodes and relationships that make up
a service’s topology – Allowing modeling of sub-topologies
Service Templates “look like nodes” enabling: Composition of applications from one or more
service templates Substitution of abstract Node types with
available service templates of the same type
Nodes Represent Components of an application or service
and their Properties. Example nodes include: – Infrastructure: Compute, Network, Storage, etc. – Platform: OS, VM, DB, Web Server, etc. – Granular: functional Libraries, Modules, etc.
Include Operations which are the management functions for the node – e.g. deploy(), start(), stop(), connect(), etc.
Export their dependencies on other nodes as Requirement and Capabilities
Relationships Represent the logical Relationships between
nodes – e.g. “hostedOn”, “connectsTo”, etc.
Describes the valid Source and Target nodes they are designed to couple – e.g. source “web application” node is designed
to “connectTo” a target “database” node Have their own Properties and Constraints
Slide origin: IBM
TOSCA: Modelling Topologies
<Definitions id="PayrollDefinitions" targetNamespace="http://www.example.com/ste" xmlns:pay="http://www.example.com/ste/Types"> <Import namespace="http://www.example.com/ste/Types" location="http://.../PayrollTypes.tosca" importType=" http://docs.oasis-open.org/tosca/ns/2011/12"/> <ServiceTemplate id="Payroll" name="Payroll Service Template"> <TopologyTemplate ID="PayrollTemplate"> <NodeTemplate id="Payroll Application" type="pay:ApplicationNodeType"> <DeploymentArtifacts> <DeploymentArtifact name="PayrollEAR" type="http://..../CSARref"> EARs/Payroll.ear </DeploymentArtifact> </DeploymentArtifacts> </NodeTemplate> <NodeTemplate id="Application Server" type="pay:ApplicationServerNodeType"> <DeploymentArtifacts> <DeploymentArtifact name="ApplicationServerImage" type="http://.../AMIref"> ami-edf2cf99 </DeploymentArtifact> </DeploymentArtifacts> </NodeTemplate> <RelationshipTemplate id="deployed_on" type="pay:deployed_on"> <SourceElement ref="Payroll Application"/> <TargetElement ref="Application Server"/> </RelationshipTemplate> </TopologyTemplate> </ServiceTemplate></Definitions>
TOSCA: Sample Definition
— Fails the “ease-of-use” test
— Does not standardise an API
But...
— Powerful and broad modelling coverage
— Workflow, locations, substitution
— Relatively mature
TOSCA
OasisCAMP
www.cloudspecs.org
Cloud Application Management Platform
http://www.oasis-open.org/committees/camp/
CAMP: Objectives and Non-Objectives
— Software
— an Orchestration Spec
— a Modelling System
But CAMP is NOT
— a REST API
— for Deployment and Management
— of Applications
It provides a flexible way to use many platforms.Interoperability.
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
Clients Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
PDP = Platform Deployment Plan
(blueprint)
— Deployment PlanCAMP: Key Concepts
DSL & REST
App ComponentType
PDP = Platform Deployment Plan
(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
— Creating Templates
POST
App ComponentTemplate
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
— ResolutionCAMP: Key Concepts
AssemblyTemplate(blueprint)
PDP = Platform Deployment Plan
(blueprint)POST
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
App ComponentTemplate
RequirementType
PlatformComponent
Template
RequirementType
App ComponentType
— Resolution
AssemblyTemplate(blueprint)
POST
AssemblyTemplate(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
App ComponentTemplate
— Resolution
200
AssemblyTemplate
POST
AssemblyTemplate(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
PlatformComponent
TemplatePlatformComponent
Template
RequirementType
App ComponentType
App ComponentType
App ComponentTemplate
— Deployment
Assembly(blueprint) Platform
Component
PlatformComponent
App Component
AssemblyTemplate(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
PlatformComponent
Template
RequirementType
RequirementType
App ComponentType
App ComponentType
App ComponentTemplate
— Deployment
POST
PlatformComponent
Template
PlatformComponent
Template
Assembly(blueprint) Platform
Component
PlatformComponent
App Component
CAMP: Key Concepts
Clients
REST
Platform
— Deployment
Assembly(blueprint) Platform
Component
PlatformComponent
App Component
AssemblyTemplate(blueprint)
PlatformComponent
Template
App ComponentTemplate
PlatformComponent
Template
POST
200
Assembly
CAMP
— depends on orchestrations supporting this
— wants standardised requirement and component types
— still needs a nice DSL
But...
— all very general
— designed to map on to a wide range of orchestrationsPaaS, TOSCA, Brooklyn, Heat
— OSS code in progressgithub.com/brooklyncentral/camp-server
CAMP: PDP/Blueprint DSL (work in progress)
Parameters
Components
Requirements
Functionsparameters & sensors
Outputs
*
*
CAMP: PDP/Blueprint DSL (work in progress)
Parameters
Functionsparameters & sensors
Outputs
*
*
components: my_app: type: WAR content: hello.war requirements: hosted_on: javaVersion: [1.7,)
Mappings(config metadata)
WaitConditions &WaitHandles
(orchestration)
Parameters
components: my_instance: type: server ram: 2048m requires: connects_to: target: my_database
Heat: New HOT DSL (also work in progress)
Functions
Outputs
HeatOrchestration
Template
Coming inHavana
Heat in Havana
Man
agem
ent
Com
posa
ble
Decl
arat
ive
Com
pone
nts
Auto
-wiri
ng
Rela
tions
hips
Exte
nsib
le
Ease
-of-
Use
Port
able
ConcurrentDistributedResilientFunctions / DI
Wrap-Up
It’s about the apps.Come get involved!
Declarative — OpenStack Heat (Grizzly) and AWS CloudFormationProcedural — Apache Whirr, Gigaspaces Cloudify, Cloudsoft BrooklynOpenStack Summit — Red Hat, RAX, IBM, HP, Cloudsoft, OASIS TOSCA and CAMPDSLs in Progress — CAMP PDP and OpenStack Heat (Havana)
