Date post: | 12-Jan-2017 |
Category: |
Technology |
Upload: | openstack-israel |
View: | 364 times |
Download: | 0 times |
Container Orchestration
RoundupTrammell Scruggs,Arthur Berezin,
GigaSpaces
Agenda◇ Orchestration 101◇ Method of comparison ◇ Tools overview
Orchestration 101
Orchestration 101◇ Common Characteristics■ Use DSL to define “blueprint”■ Execute a process based on input from the blueprint ■ Pass context information between the deployed
entities
◇ Different assumptions lead to different approaches ■ Application Architecture■ Infrastructure ■ Scope of automation
Method of Comparison◇ Same application requirements ◇ “Production grade” deployment ◇ Broken into three main groups■ Pure Play – Cloudify/TOSCA,
Terraform■ Container Centric – Kubernetes■ Infrastructure Centric - OpenStack Heat
◇ PaaS: CloudFoundry, OpenShift, DEIS◇ Config Management: Chef, Puppet, Ansible, Salt◇ Cloud Application Management: Apache Brooklyn, Juju◇ Cloud management platforms: RH CloudForms, Scalr◇ Cluster schedules: Mesos, Fleet, Yarn, Nomad
This is by no means an exhaustive comparison!
A Lot of Relevant Tools Which We Won’t Cover Today :(
The Test Application Test Application
◇ Manage dependencies◇ Reproducible ◇ Cloneable◇ Recoverable◇ Updateable
Common Requirements
◇ Setup monitoring and log collection ◇ Manual/Auto healing ◇ Manual/Auto scaling ◇ “Day 2”■ Backup and restore ■ Update code ■ Infrastructure upgrades and patches
Beyond Just Setup
“Pure-Play” Orchestration
Terraform by Hashicorp
Introduction to Terraform◇ Part of the Hashicorp “ecosystem”◇ Command line tool ◇ Simple (in a good way) configuration◇ Extendable using plugins
Solution Overview
◇ Single top level configuration file ◇ Create all infrastructure install and
configure application with scripts.
Sample Configuration resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] }}
Deployment Management
◇ Option 1: Static Allocation/Provisioning in the configuration file.
◇ Option 2: Third-party Tool■ Consul/Surf■ Other, e.g. AWS Autoscale
Cons: ◇ Configurations are not
portable across cloud providers
◇ Outsources advanced configuration management to other tools
◇ Context awareness is not built in
Terraform: Pros and Cons Pros: ◇ Simple & elegant◇ Solid Openstack
support◇ IaaS & Framework
neutral◇ Display Plan before
deployment◇ Idempotency◇ Hashicorp stack
TOSCA/Cloudify
TopologyNodes,
relationships, requirements &
capabilities
Workflows
Lifecycle operations
implementation (install, scale…)
PolicyEvents,
conditions, action
mapping
Topology Orchestration Spec for Cloud Applications
What is TOSCA?
What is TOSCA?◇ Standard for Cloud Orchestration◇ Describes how to map application
requirements to infrastructure capabilities
ApplicationBlueprint(TOSCA)
IaaSPlugins
ContainerPlugins
Conf MgmtPlugins
● Provision● Configure● Monitor● Manage
Monitoring &Alarming
Cloudify: an Open Source Implementation of TOSCA
Solution Overview◇ Single blueprint file with peripheral
scripts and plugins◇ Create all infrastructure install and
configure application with scripts◇ Describe future application
management and workflows◇ Relationships describe which code to
execute when one node event affects another node.
nodecellar_container: type: nodecellar.nodes.NodecellarContainer properties: Name: nodecellar image: Repository: nodecellar relationships: - type: node_contained_in_nodejs target: nodejs_host
Sample Configuration
nodejs_host: type: cloudify.openstack.nodes.Server interfaces: cloudify.interfaces.monitoring: start: implementation: diamond.diamond_agent.tasks.add_collectors inputs: collectors_config: default: CPUCollector: {} MemoryCollector: {} LoadAverageCollector: {} DiskUsageCollector: config: devices: x?vd[a-z]+[0-9]*$ NetworkCollector: {}
Sample Configuration
Deployment Management◇ Manager is aware of node instances’
state◇ logging and monitoring◇ Policy engine handles metric streams◇ Manager auto-executes workflows in
response to lifecycle requirements
◇ Built in workflows ■ Install■ Uninstall■ Heal■ Scale■ Execute_operation■ (Single common ‘cfy install’ in 3.4 )◇ Dependency awareness through graph
navigation◇ Remote/Local execution
Handling Post Deployment through Workflow & Policies
Cons: ◇ Complex◇ Delay in full support of
TOSCA spec◇ Not enough
implementations - user needs to write most of it
TOSCA/Cloudify: Pros and Cons
*Implementation specific
Pros: ◇ Infrastructure &
Framework neutrality ◇ Complete Life Cycle
Management ◇ Handles
Infrastructure & Software
◇ Post deployment handling*■ Monitoring■ Logging
◇ Can be tied to any peripheral system
Container OrchestrationKubernetes
Container deployment management ◇ Pods: a group of containers usually run together◇ Replication controller: ensures state of pod◇ Networking: pods are networked together◇ Service: Load balanced endpoint for a set of
pods
Quick Intro to K8s
K8s Architecture
Solution Overview◇ Single configuration file ◇ Uses pools of infrastructure resources◇ Describe future application
management and workflows◇ Relationships are maintained through
Pods
apiVersion: v1kind: Servicemetadata: name: nodecellar-servicespec: selector: app: nodecellar ports: - protocol: TCP port: 3000 targetPort: 3000 nodePort: 30000 type: NodePort
Sample ConfigurationapiVersion: v1kind: ReplicationControllermetadata: name: nodecellarspec: replicas: 2 selector: app: nodecellar template: metadata: name: nodecellar labels: app: nodecellar spec: containers: - name: nodecellar Image: nodecellar workingDir: / command: ["bash","start.sh"] ports: - containerPort: 3000 hostIP: 0.0.0.0
Cons: ◇ Not a lot of infrastructure
focus◇ Limited Openstack
support (mostly Rackspace)
◇ No topology awareness◇ Complex setup
Kubernetes: Pros and Cons Pros: ◇ (almost) zero
configuration autoheal
◇ Out of the box load balancer
◇ Simple scaling◇ Advanced behaviors
are pretty simple to implement
Infrastructure Centric
What Is Heat?
Provides a mechanism for orchestrating OpenStack resources through the use of modular templates
Heat Architecture
OpenStack APIs
Solution Overview
Output:Replica set node hostsssh-key, private ip to the init node
Input:# of replicas per shardInput:
# of nodeJS instances# MongoConfig hosts# Mongo shards hosts
Output:mongos node hostsApp EndPoint = Load-Balancer IP/path
Output:mogocfg node hosts
Input:# of config instances
resources: secgroup: type: OS::Neutron::SecurityGroup properties: name: str_replace: template: mongodb-$stackstr-secgroup params: $stackstr: get_attr: - stack-string - value rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 - protocol: tcp port_range_min: 27017 port_range_max: 27019
Infrastructure setup
mongo_host: type: OS::Nova::Server properties: name: str_replace: template: $stackprefix-$stackstr params: $stackprefix: get_param: stack-prefix $stackstr: get_attr: - stack-string - value Image:
get_param: image flavor: get_param: flavor security_groups: - get_param: security_group
Create Compute Instances
mongodb_peer_servers: type: "OS::Heat::ResourceGroup" properties: count: { get_param: peer_server_count } resource_def: type: { get_param: child_template } properties: server_hostname: str_replace: template: '%name%-0%index%' params: '%name%': { get_param: server_hostname } image: { get_param: image } flavor: { get_param: flavor } ssh_key: { get_resource: ssh_key } ssh_private_key: { get_attr: [ssh_key, private_key] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version }
Create MongoDB Replica Servers
server_setup: type: "OS::Heat::ChefSolo" depends_on: - mongodb_peer_servers properties: username: root private_key: { get_attr: [ssh_key, private_key] } host: { get_attr: [mongodb_peer_servers, accessIPv4, 0] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version } node: mongodb: ruby_gems: mongo: '1.12.0' bson_ext: '1.12.0' bind_ip: { get_attr: [mongodb_peer_servers, privateIPv4, 0] } use_fqdn: false replicaset_members: { get_attr: [mongodb_peer_servers, privateIPv4] } config: replset: myreplset run_list: [ "recipe[config_replset]" ]
Configure the Replica Servers
nodestack_chef_run: type: 'OS::Heat::ChefSolo' depends_on: nodestack_node properties:
... node: nodejs_app:
... deployment: id: { get_param: stack_id } app_id: nodejs run_list: ["recipe[apt]", "recipe[nodejs]", "recipe[ssh_known_hosts]", "recipe[nodejs_app]"] data_bags: nodejs: id: { get_param: stack_id } nodejs_app: password: { get_attr: [nodejs_user_password, value] } deploy_key: { get_param: deploy_key } database_url: str_replace: template: 'mongodb://%dbuser%:%dbpasswd%@%dbhostname%' params: '%dbuser%': { get_param: database_username } '%dbpasswd%': { get_param: database_user_password } '%dbhostname%': { get_param: db_server_ip }
Create NodeJS Container
Cons: ◇ OpenStack Only ◇ Software configuration is
limited◇ Lack of built-in workflow
abstraction ◇ Post deployment
orchestration is limited■ Requires integration with other tools/ projects
Heat: Pros and Cons Pros: ◇ Native To OpenStack◇ Built-in mapping of
most OpenStack infrastructure resource types
So, what’s the Right Tool for me? ◇ Are you hell bent on containers? OpenStack? ◇ Do you have legacy workloads? ◇ Do you consider infrastructure resources as part of
the process? ◇ Do you have a heterogenous environment? ◇ Do you want the same tool to also handle post
deployment?
The Tools Presented Are Not Necessarily Mutually Exclusive!
Final Words
The Only Constant is Change