© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Deploying OpenStack for IaaS
with the Cisco Edition
Robert Starmer
BRKAPP-2031
3 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Agenda
Cloud Computing Concepts
‒ Cloud Abstractions
‒ Cloud Management Framework Architecture
Cloud Concept Realization
‒ OpenStack
‒ OpenStack Cisco Edition
‒ Building your own Cloud
The rest of the Cloud
‒ IT Management Systems
‒ Application Deployment Tools
3
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Business Models
4 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
5 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
5 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
Nebulous
6 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
6 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
Ethereal
7 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
7 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
Foggy
8 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Cloud Defined
8 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
Public Private Hybrid Community
Deployment
Models
Service
Models Software as a Service (SaaS)
Platform as a Service (PaaS)
Infrastucture as a Service (IaaS)
Essential
Characteristics On-Demand Self Service
Broad Network Access
Resource Pooling
Rapid Elasticity Measured Service
IT resources and services that are abstracted from the underlying infrastructure provided “on-
demand” and “at scale” in a multi-tenant environment
9 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Key Orchestration Components
• Portal/API(s)
• Policy/Rules/Workflow Engine
• “Domain” Managers (also Policy/Rules/Workflow based)
• Integration APIs between “Domains”
9 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
10 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
General Business Use Cases
IaaS
‒ CaaS
‒ Web2.0/AWS
‒ AppInfra
PaaS
‒ Web2.0 “No Infra”
‒ TargetedApp
• SaaS
Catalog
Unified Portal
• Hybrid
DR
App Mobility
10 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
11 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Click to edit Master text styles
11 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
IaaS
Service Model Characteristics Example solutions
IaaS CaaS/ VMM • Self-service mgmt of virtual compute.
• Perception of low network & storage relevance
• Simple tenancy model per VM.
• CLM+ESX
• DynamicOps+Vsphere
• Cloupia + XenServer
• MSSC + HyperV
Web2.0 • VM mgmt at scale.
• API-driven compute automation: Spin up and shut down
webservers as load changes
• App dev’t model more efficient
• Load Balancing, segmentation, simple security.
• Tenancy = in theory, infinite.
• OpenStack
• CloudStack
AppInfra • Compute focused Internal cloud solutions
• Often used to scale specific consumer facing apps
• App scale of Web2.0
• Simple tenancy: 10s or 100s.
• OpenStack (e.g., Webex)
• CLM
• IAC
12 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Click to edit Master text styles
12 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
PaaS
Service Model Characteristics Example solutions
PaaS No Infra
Apps
• No VM or direct storage view, everything is abstracted
away
• Focus is on scale out.
• MVC: Model View Controller – web app dev’t paradigm
• Model = data store (db)
• Controller = App logic
• View: presentation to the consumer.
• each of the 3 elements scale independently
• Heroku
• CloudFoundry
• AppFog
• EngineYard
Targeted
App
• A platform for developing S/W for you to use rather than
for others to use
• Often focused on an app family (E.g. business
management, gaming infra, etc.)
• Google app engine &
Google apps
• SalesForce
“Force.com”,
• WordPress
13 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 13 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
SaaS
Service Model Characteristics Example solutions
SaaS Catalog-
driven
• E.g., SP providing SaaS
• Service offers bundled at the application layer
• Deployment automation for tenant scale
• Focus is on managing access to the app
• Parallels
• CCP+rPath+CPO+AS
Unified
Portal
• E.g., Enterprise view.
• Aggregate multiple services
• Single Sign-on management
• Greater potential for custom development
• JamCracker
• Citrix SaaS
14 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 14 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
Hybrid
Service Model Characteristics Example solutions
Hybrid DR/IaaS • Classic multi-DC issues (disk sync, etc.)
• Infrastructure-to-infrastructure synchronization
• VM level view
• Migration possible, but potentially slow (Large VM size
issue)
• vCloud Director, ‘multi-
cloud’ models
App
Mobility,
Data
Mobility
• Suitable for Web 2.0 application models
• MVC or similar app dev’t
• App level view
• Migrate the app rather than the VM
• How do you migrate the Data?
• Real hybrid use case: takes into account application
• private CloudFoundry-
>hosted
CloudFoundry?
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 15
“Here be
Dragons”
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
Welcome to OpenStack
The Cloud needs an Open Source platform to achieve Internet Scale:
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack: A Brief History
nebula.nasa.gov
• NASA Launches Nebula
‒ One of the first cloud computing platforms built
by the Federal Government for the Federal
Government
• March 2010: Rackspace Open Sources
Cloud Files software, aka Swift
• May 2010: NASA open sources
compute software, aka “Nova”
• June 2010: OpenStack is formed
• July 2010: The inaugural Design
Summit
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Community
160 and counting
+ &
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Vision
Seamless Cloud Interoperability
Public Clouds Private Clouds
Community Clouds
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Introduction
• A Cloud Platform
‒ A collection of interrelated software components delivering capabilities to build and
manage cloud infrastructure.
• A global community of developers devoted to innovation and openness
• Flexibility in deployment and features
• Standards for broad deployment
• No fear of vendor “lock-in”
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Terminology
• Instance- Running virtual machine
• Image- Non-running virtual machine, multiple formats (AMI, OVF, etc.)
• Application Programming Interface (API)- Interface for computer programs
• Message Queue- Acts as a hub for passing messages between daemons
• Volume- Provides persistent block storage to instances
• Project- aka Tenants, provides logical separation among cloud users
• Flavors- Pre-created bundles of compute resources
• Fixed IP- Associated to an instance on start-up, internal only
• Floating IP- Public facing IP address
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Core Projects
OpenStack Compute (Nova) Software to provision virtual machines on standard server hardware at massive scale
OpenStack Object Storage (Swift) Software to reliably store billions of objects distributed across standard server hardware
OpenStack Image Service (Glance) Services for discovering, registering, and retrieving virtual machine images
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Core Projects Cont..
OpenStack Dashboard (Horizon) A self-service web portal to allow administrators and users to manage OpenStack resources
OpenStack Identity (Keystone) Provides “unified authentication” across all OpenStack projects and integrates with 3rd party authentication systems
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Core Projects Cont..
OpenStack Network Service (Quantum) Provides “network connectivity as a service” between devices managed by other OpenStack services
OpenStack Block Storage (Cinder) Provides persistent block storage to guest VMs
© 2012 Cisco and/or its affiliates. All rights reserved. TECDCT-2242 Cisco Public
OpenStack Incubation Projects
OpenStack Monitoring and Metering (Ceilometer) Infrastructure to collect measurements within OpenStack
OpenStack Orchestration Service (Heat) Service to orchestrate multiple composite cloud applications using the AWS CloudFormation template format
Many Other Community Projects http://wiki.openstack.org/Projects http://openstack.org/projects/
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
OpenStack pieces, Interaction
http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
27 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 27 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Cloud Orchestration Stack Overview
Compute Network Storage
Physical Infrastructure Element Management
Compute Network Storage
Compute API Network API Storage API
Ph
ysic
a
l M
gm
t.
Vir
tua
l
Mgm
t.
Intelligent Placement, Resource, Consumption, Event Management
User/Admin Portal System API
Service
Catalog
Federated
Resource DB
Service
Assurance
Manager
Billing Integration
SA API
Clo
ud
Mgm
t. CRM Integration
CMDBIntegration
Help
Desk
SA Integration
28 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 28 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
OpenStack Stack Touchpoints
Compute Network Storage
Physical Infrastructure Element Management
Compute Network Storage
Compute API Network API Storage API
Ph
ysic
a
l M
gm
t.
Vir
tua
l
Mgm
t.
Intelligent Placement, Resource, Consumption, Event Management
User/Admin Portal System API
Service
Catalog
Federated
Resource DB
Service
Assurance
Manager
Billing Integration
SA API
Clo
ud
Mgm
t. CRM Integration
CMDBIntegration
Help
Desk
SA Integration
29 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
User Models
• Who can best use OpenStack?
• Need dynamic workload provisioning, preferably API driven
• Applications most likely to leverage a “Web2.0” deployment model
• Understanding of the need for development resources as a part of the
Cloud Infrastructure team
29
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
DevOps for Openstack Deployment
30
31 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
How do I deploy OpenStack?
• Manual install and configuration
• Scripted installers
• DevOps processes
31
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 32
DevOps – Development, Deployment, and Operations
Agile/Extreme/Lean/Etc. application
development expect rapid turn from
development->test->production
Model for Deployment built into the
development/test lifecycle
‒ Unit test
‒ Continuous Integration
Move from semi-annual release to
daily or weekly releases
Some iterate ~40x/day dev-
>production!
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 33
GIT
A “modern” Source Code Management (SCM) system
Uses a pointer paridigm rather than patch model
Similar but different to RCS/SCCS/CVS/SVN
Biggest end user difference is that branches and merges become easier
Driven by the need for concurrent development of very large projects (Linux
Kernel dev community)
OpenSource DevOps tools
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 34
Why Git?
OpenStack community selected Git as the repository of record for source, and
specifically github.com (not the owner of Git, just a _VERY_ heavy user)
Git integrates well with many of the development workflows and processes used
in Agile/Extreme/Lean class development processes
Git works well with large distributed teams working on the same codebase
Try it you’ll like it
More info: http://git-scm.com, http://git-scm.com/book
OpenSource DevOps Tools
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 35
Simple Puppet
One of the new breed of “DevOps” tools – Data drive Operations
Others include Chef, Ansible, JuJu, etc.
Development driven operationalization of systems
Developers write the app
Developers write the test
Developers write the deployment
Developers write the upgrade
Developers wrote the operational model
OpenSource DevOps Tools
36 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Configuration Interactions
• Quantum auth config:
[filter:authtoken]
paste.filter_factory =
keystone.middleware.auth_token:filter_factory
auth_host=192.168.25.10
auth_port = 35357
auth_protocol = http
admin_tenant_name=services
admin_user=quantum
admin_password=quantum
• quantum_config { 'auth_strategy': value =>
$auth_strategy
• Nova auth config:
[filter:authtoken]
paste.filter_factory =
keystone.middleware.auth_token:filter_factory
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:35357/v2.0
admin_tenant_name = services
admin_user = nova
admin_password = nova_pass
• nova_config { 'auth_strategy': value =>
$auth_strategy
36
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Cisco Edition Reference Architecture
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 38
Reference Systems Model
Single Rack
‒ Up to 36 1RU rack servers
‒ 2-4 TOR devices
‒ Local storage for block and object storage
Near term target for Cisco Validated Design (anticipate Fall 2013)
Intended to start with sub rack scale, and grow to 10s-100s of racks
http://docwiki.cisco.com/wiki/OpenStack:Reference_Architectures:UCS_C2xx_M
3
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 39
Build Following Reference Architecture
Multi-tenant (multi-project), compute, network, storage
Use Quantum for network, L3Agent (virtual router) for L3 segregation
Per tenant network, L3/NAT for segregation
shared “public” network
Non-HA control plane
Feb 2013 release
40 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Build Process
First, build a build node (often a manual process)
Second, download (clone) the CiscoSystems github repository
Run the install script (modify the default parameters)
Run puppet, configure puppet master, mysql, cobbler
Power-on the rest of the system, let cobbler and puppet work their magic
40
Let’s Build one!
42 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Why all of this complexity?
42
43 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Node type distribution
43
Storage Node
(future)
SWIFT
Control Node
Nova
Quantum
Glance
Keystone
Horizon
AMQP(rabbitmq)
Control Node
Compute Node
Nova
Cinder
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 44
Collect the pieces you need
• Reference Architecture Hardware
‒ http://cisco.com/go/openstack
• Ubuntu Linux OS (best current support)
‒ http://releases.ubuntu.com/precise, 12.04.1 x86_64 server version
• Access to the internet from the cloud system(s)
‒ https://github.com/CiscoSystems
‒ ftp://ftpeng.cisco.com/openstack/cisco
‒ http://us.archive.ubuntu.com/
45 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Puppet Modules - OpenStack
Puppetlabs Github is source of record
‒ puppetlabs-openstack
‒ puppetlabs-nova
‒ puppetlabs-quantum
‒ puppetlabs-keystone
‒ puppetlabs-rabbitmq
‒ puppetlabs-glance
https://github.com/puppetlabs
Cisco validated variants (and quantum work)
https://github.com/CiscoSystems
45
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 46
Collect the address information needed
• IP addresses for management interfaces:
‒ Build node, control node, compute node(s)
• MAC addresses from control/compute nodes
• DNS information (or at least upstream dns server)
• Determine Quantum based network model
47 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 47
TOR-a
Build
Control(s)
Compute(s)
Not Shown: OOB Network
FI/FEX
Upstream net
192.168.28.252
admin/cisco
E 1/1-10: VLAN 101
VLAN gw 192.168.101.1
192.168.101.240
Power: 192.168.28.10 admin/pass
Mac: 70:CA:9B:CE:35:92
192.168.101.230
Power: 192.168.28.17 admin/pass
Mac: 70:CA:9B:CE:2E:EA
192.168.101.220
Power: 192.168.28.16 admin/pass
Mac: 00:10:18:BE:E9:10
OpenStack Cisco Edition Demo
‒ Build node (pre-built)
‒ Cobbler bare metal install
‒ Puppet configuration
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 48
Install the first node (manual)
• Use UCSM or CIMC interface to provide remote KVM and virtual CD
• Mount the ISO, and build the node. Default options, LVM against the local
RAID configuration, OpenSSH installed.
• When build is complete, add base build packages:
‒ apt-get install git puppet ipmitool python-passlib python-jinja2 python-yaml –y
• Get the puppet install and build manifests:
‒ git clone https://github.com/CiscoSystems/folsom-manifests -b multi-node
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 49
Setup basic parameters
• Copy the folsom-manifests/manifests directory to /etc/puppet/manifests
• Load the puppet modules: /etc/puppet/manifests/puppet-modules.sh
• Edit the site.pp file with the addresses collected earlier
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 50
Run puppet on the build/puppetmaster node
• puppet apply –v /etc/puppet/manifests/site.pp
• puppet plugin download
• “reset” your environment:
• /etc/puppet/manifests/reset_nodes.sh
• Wait ~15 minutes. Log into your control node:
• ssh localadmin@control or https://control_node_ip
51 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Demo - Deploy OpenStack Platform
51
52 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
That’s the basics – now for the network
• Leveraging OpenVirtualSwitch for Linux based switching
• Leveraging Linux Iptables for firewall/router
• Using DNSMasq for DHCP and DNS proxy services
• For network, there are two deployment choices
‒ Nova-network, proven, in production
‒ Quantum, lots of testing, still not common in production use
52
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53
OpenStack Network History NASA Nebula cloud
Principally VPN session to “VLAN” segregated network
FlatDHCP model per tenant with VPN outside->in access
NOVA
Flat model
Flat + DHCP/iptables/meta-data
Flat+DHCP on each compute node “multi-host”
VLAN, like Flat, but with more than “one” target
Quantum
Break network out of Nova
NOVA includes more than “network”
DHCP
L3
IPAM
Metadata (Cloud-Init)
Quantum adding them
Melange merged - Adds IPAM
L3 agent extracted, but missing capabilit
Service Insertion coming
Addresses L3 HA
Addresses missing service models
VPN, L3/HA, LB, FW, etc.
54 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Quantum – Network Models
54
55 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Single Flat or Provider network
55
• Simple model
• Equivalent to nova-network VLAN
• No dhcp, metadata, NAT, etc.
• All tenants/projects see each other
• Router managed by something
other than OS
56 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Shared Flat Networks
• Provides a model to
exponse multiple L2
domains to end users
• Can provide some tenante
segregation
• Still no
dhcp,metadata,NAT
56
57 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Mixed Flat and Private Networks
• Private networks (big difference in
this example), provide:
‒ DHCP
‒ metadata
• Shared newtork in this model is as
before
57
58 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Provider Router and private networks
• Provider router:
‒ Adds NAT
• Other networks as before
58
59 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Per Tenant routers with Private networks
59
60 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Demo
60
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 61
Great. Now What?
• Load Images
• Onboard Users
• Create additional tenants
• Deploy VMs
• Assign and manage quotas
• Connect in to billing/chargeback mechanisms
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Integrating with other “management”
systems
© 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 63
Example: Leverage Cisco Intelligent Automation for Cloud
Service catalog, user and quota management
Help desk
Billing accounting integration
Hybrid cloud management capabilities
OpenStack cloud acceleration pack available in 3.2 release
64 © 2012 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Video - Overview of IAC Multi-Cloud with OpenStack
64
QuickTime™ and a decompressor
are needed to see this picture.
© 2013 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public
Call to Action
• Visit the Cisco Campus at the World of Solutions to experience Cisco innovations in action
• Get hands-on experience attending one of the Walk-in Labs
• Schedule face to face meeting with one of Cisco’s engineers
at the Meet the Engineer center
• Discuss your project’s challenges at the Technical Solutions Clinics
65
© 2013 Cisco and/or its affiliates. All rights reserved. BRKAPP-2031 Cisco Public 66