Xen and CloudStack
Ewan Mellor
Director, Engineering, Open-source Cloud Platforms
Citrix Systems
• What is CloudStack?
• Move to the Apache Foundation
• CloudStack architecture on Xen
• The future for CloudStack
• Developing on CloudStack
Agenda
• Infrastructure-as-a-Service software
• For people to build Amazon-style clouds
– either as a public service,
– or as a private cloud within your own organization.
CloudStack is…
CloudStack is…
Compute
XenServer Oracle VM KVM vSphere Bare metal
Hypervisor
Storage
Local Disk iSCSI NFS
Fiber
Channel Swift
Block & Object
Network
Network
Type Isolation
Load
balancer Firewall VPN
Network & Network Services
CloudStack
Apache Foundation
(incubating)
2008-present
Java
Fairly centralized
Complete solution
CloudStack versus OpenStack
OpenStack
OpenStack Foundation
(founding in progress)
2010-present
Python
Aggressively distributed
More of a framework
• Citrix donated the CloudStack code to the
Apache Software Foundation and Apache
Incubator
• The project now needs to prove that it can
operate openly and independent of Citrix
• If it can do that, it can be accepted as a
top-level Apache project.
Apache incubation
Most CloudStack users choose Xen or XenServer
It is the combination that works the best!
CloudStack and Xen
Xen
VMware
KVM
• A cluster of management servers
• An orchestration engine
• State and user management
• Region, zone, pod, cluster model
• A large collection of effectors
• System VMs
• Usage reporting
• User interface
CloudStack is…
Router
L3 core switch
Access layer
switches
Servers
Management
cluster
Secondary
Storage
Pod 1 Pod 2 Pod 3 Pod N
Load balancer
Users
Pod 1
Host 2
Cluster 1
Host 1
Primary
storage
L3 switch
Secondary
storage
L2 switch
Two types of storage
• Stores virtual disks
• Configured at cluster level
• Close to hosts for better performance
• Requires high IOPS
Primary storage
• Stores templates, ISOs and snapshots
• Configured at zone level
• Zone can have one or more
• High capacity, low cost
Secondary storage
Router
L3 core switch
Access layer
switches
Servers
Management
cluster
Secondary
Storage
Pod 1 Pod 2 Pod 3 Pod N
Load balancer
Users
• Choice of network isolation – Physical, VLAN, L3 (anti-spoof), overlay (GRE)
• Multiple networks – Shared networks, project networks
• IPAM / DHCP
• Gateway (inc VRRP)
• VPN, stateful firewall, NAT, port forwarding
• Traffic monitoring
• Load balancing
• User-data / password-change services
Networking features
Virtual network topologies
Public network
Guest network
DHCP, DNS, NAT, LB, VPN
10.1.1.1
Guest
VM 1
10.1.1.3
Guest
VM 2
10.1.1.4
Guest
VM 3
10.1.1.5
Guest
VM 4
CS
Virtual
Router
Public network
DHCP, DNS
10.1.1.1
Guest
VM 1
10.1.1.3
Guest
VM 2
10.1.1.4
Guest
VM 3
10.1.1.5
Guest
VM 4
NetScaler
LB
CS Virtual Router provides network services External devices provide network services
CS
Virtual
Router
SRX
firewall
Guest network
Public network
Multi-tier virtual network topology
DHCP, DNS, user-data
10.1.1.1
Web VM
1
10.1.1.3
Web VM
2
10.1.1.4
Web VM
3
10.1.1.5
Web VM
4
Netscaler
LB
CS
Virtual
Router
CS
Virtual
Router
10.1.2.21
10.1.2.18
10.1.2.38
10.1.2.39
10.1.2.31
App VM
1 10.1.3.21
10.1.2.24
App VM
2 10.1.3.45
10.1.3.24
DB VM 1
DHCP, DNS,
user-data
DHCP, DNS user-data, NAT, VPN
Public network
Guest network Guest network Guest network
CS
Virtual
Router
SRX firewall
Router
L3 core switch
Access layer
switches
Servers
Management
cluster
Secondary
Storage
Pod 1 Pod 2 Pod 3 Pod N
Load balancer
Users
Router
L3 core switch
Access layer
switches
Servers
Management
cluster
Secondary
Storage
Pod 1 Pod 2 Pod 3 Pod N
Load balancer
Users
System VMs
Customer VMs
18
Driver domains
Xen Hypervisor
Control domain (dom0)
Host HW
Guest VMn
Apps
Memory CPUs I/O
HW Drivers
PV Back Ends PV Front Ends
Driver Domain e.g. • Disk • Network
HW Driver
PV Back End
Kernel Guest OS
The road ahead
• Disentangle some core components
– Make it easier for people to work on
subsystems
– Make it easier to scale the API layers
The future for CloudStack
… … …
S3 API
VM
S3 API
VM
S3 API
VM
Object store
(e.g., NetApp, Hadoop, …)
S3 API
NFS, CIFS,
HDFS, 3rd-
party object
store API, …
S3 API system VM
• Absorb new changes from Xen and XS
– Storage migration
– The Windsor architecture
– I/O subsystem changes
The future for CloudStack
• New object storage systems
– Caringo CAStor
– HDFS
– Riak CS
• New block storage systems
– GlusterFS?
– Ceph?
• Open vSwitch / OpenFlow / VXLAN
The future for CloudStack
DevCloud
DevCloud – What is inside?
• VirtualBox VM
– Xen Cloud Platform
– Ubuntu 12.04 dom0 (aka Kronos)
• JVM, Tomcat, developer tools
• NFS server
• MySQL
– Working, pre-configured CloudStack
– System VMs
– Tiny Linux template
Why?
• Eliminates
– need for another machine / hypervisor
– Primary and secondary storage
– Networking, VLAN, etc
– Install and configuration of environment
• Disposable changes
– Revert to snapshot
Run
• Start the DevCloud VM from VirtualBox UI
• Access the CloudStack UI
– http://localhost:8080/client
• Access the dom0
– ssh root@localhost –p 2222
• From the UI, enable the zone
– System VMs will start up
CloudStack architecture
Management Server Cluster
Replica
Infrastructure
Resources
User API
Admin API
Load
Balancer
Management
Server
Management
Server
MySQL
MS is stateless. MS can be deployed as physical server or VM
Single MS node can manage up to 10K hosts. Multiple nodes can be deployed for scale or redundancy
Replication
• Hosts • Servers onto which services will be
provisioned
• Primary Storage • VM disk storage
• Cluster • A grouping of hosts and their associated
storage
• Pod • Collection of clusters in the same failure
boundary
• Network • Logical network associated with service
offerings
• Secondary Storage • Template, snapshot and ISO storage
• Zone • Collection of pods, network offerings and
secondary storage
• Management Server Farm • Management and provisioning tasks
Components
Zone
CloudStack Pod
Cluster
Host
Host
Network
Primar
y
Storag
e
VM
VM
CloudStack Pod
Cluster Seconda
ry
Storage
Pod 1
….
Cluster N
L2
Host 2
Cluster 1
Deployment Architecture
Host 1
Hypervisor is the basic unit of scale.
Cluster consists of one ore more hosts of same hypervisor
All hosts in cluster have access to shared (primary) storage
Pod is one or more clusters, usually with L2 switches.
Availability Zone has one or more pods, has access to secondary storage.
One or more zones represent cloud
Primary
Storage
Zone 1
….
L3
Secondar
y
Storage
Pod N
Management
Server
Cluster
Internet
DevCloud step-by-step
Overview
• Install Virtual Box
• Download DevCloud virtual machine
• Install DevCloud VM
• Run the VM
• Login to the CloudStack GUI
• Start a CloudStack VM.
Virtual Box
• Download / install from Oracle
• Ensure that VT-d and virtualization
extensions are enabled in your laptop
CPU
• Recommend at least 2 GB for DevCloud
VM
DevCloud VM
• Download from http://download.cloud.com/templates/devcloud/DevCloud.ova
• Latest install instructions here:
http://wiki.cloudstack.org/display/COMM/DevCloud
• Import DevCloud into VirtualBox
– Change memory allocation to 2 GB
• Snapshot for known good state