Post on 16-Jan-2015
description
transcript
XenServer ResourceXenServer Resource
Agent ManagerAgent Manager
API LayerAPI LayerEC2EC2 CloudStackCloudStack
Virt
ual M
achi
ne M
anag
erVi
rtua
l Mac
hine
Man
ager
KVM Resource
KVM Resource
vSphere ResourcevSphere Resource
SRX Resource
SRX Resource F5 ResourceF5 Resource NetScaler
ResourceNetScaler Resource
OVM Resource
OVM Resource
Access ControlAccess Control
Stor
age
Man
ager
Stor
age
Man
ager
Net
wor
k M
anag
erN
etw
ork
Man
ager
Cons
ole
Prox
y M
anag
erCo
nsol
e Pr
oxy
Man
ager
Snap
shot
Man
ager
Snap
shot
Man
ager
Tem
plat
e M
anag
erTe
mpl
ate
Man
ager
Asyn
c Jo
b M
anag
erAs
ync
Job
Man
ager
…
Management Server
Kernel- Drives long running VM
operations- Syncs between resources
managed and DB- Generates events
Resource Management
Cluster Management
JobManagement DB
UI Cloud Portal CLI
Other Clients
Job Queue
Deployment Planning
Network Configurations
Network Elements
Hypervisor Gurus
DatabaseAccess
Alert & EventManagement
Plug
in A
PI
Hypervisor Resources
Network Resources
Storage Resources
ImageResources
SnapshotResources
REST API
OAM&P API End User API EC2 API Pluggable Service API EngineOther APIs
Security Adapters
Account Management Connectors
ACL & Authentication- Accounts, Domains, and Projects- ACL, limits checking
Services API
Serv
ices
API
Console Proxy Management
Template Access
HA
Usage Calculations
Additional Services
Event BusMessage Bus
Kernel Module
• Understands how to orchestrate long running processes (i.e. VM starts, Snapshot copies, Template propagation)
• Well defined process steps• Calls Plugin API to execute functionalities that
it needs
Plugins
• Various ways to add more capability to CloudStack
• Implements clearly defined interfaces• All operations must be idempotent• All calls are at transaction boundaries• Compiles only against the Plugin API module
Anatomy of a Plugin
ServerResource- Optional. Required if
Plugin needs to be co-located with the resource
- Implements translation layer to talk to resource
- Communicates with server component via JSON
ServerResource- Optional. Required if
Plugin needs to be co-located with the resource
- Implements translation layer to talk to resource
- Communicates with server component via JSON
Rest API- Optional. Required only if needs to expose configuration API to admin.
Plug
in A
PI
Data Access Layer
Implmentation
Anatomy of a Plugin
• Can be two jars: server component to be deployed on management server and an optional ServerResource component to be deployed co-located with the resource
• Server component can implement multiple Plugin APIs to affect its feature
• Can expose its own API through Pluggable Service so administrators can configure the plugin
• As an example, OVS plugin actually implements both NetworkGuru and NetworkElement
Plugin Interfaces Available• NetworkGuru – Implements various network isolation technologies
and ip address technologies• NetworkElement – Facilitate network services on network elements
to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc)• DeploymentPlanner – Different algorithms to place a VM and
volumes.• Investigator – Ways to find out if a host is down or VM is down.• Fencer – Ways to fence off a VM if the state is unknown• UserAuthenticator – Methods of authenticating a user• SecurityChecker – ACL access• HostAllocator – Provides different ways to allocate host• StoragePoolAllocator – Provides different ways to allocate volumes
Adding a Plugin to CloudStack
• Components are configured though components.xml
• Supports DAO, Manager, and Adapter patterns• Open to other component frameworks (OSGi a
possibility)
Components.xml Example<components.xml> <system-integrity-checker class="com.cloud.upgrade.DatabaseUpgradeChecker"> <checker name="ManagementServerNode" class="com.cloud.cluster.ManagementServerNode"/> <checker name="EncryptionSecretKeyChecker" class="com.cloud.utils.crypt.EncryptionSecretKeyChecker"/> <checker name="DatabaseIntegrityChecker" class="com.cloud.upgrade.DatabaseIntegrityChecker"/> <checker name="DatabaseUpgradeChecker" class="com.cloud.upgrade.PremiumDatabaseUpgradeChecker"/> </system-integrity-checker> <interceptor library="com.cloud.configuration.DefaultInterceptorLibrary"/> <management-server class="com.cloud.server.ManagementServerExtImpl" library="com.cloud.configuration.PremiumComponentLibrary"> <adapters key="com.cloud.storage.allocator.StoragePoolAllocator"> <adapter name="LocalStorage" class="com.cloud.storage.allocator.LocalStoragePoolAllocator"/> <adapter name="Storage" class="com.cloud.storage.allocator.FirstFitStoragePoolAllocator"/> </adapters> <pluggableservice name="VirtualRouterElementService" key="com.cloud.network.element.VirtualRouterElementService" class="com.cloud.network.element.VirtualRouterElement"/> </management-server></components.xml>
Kernel
Sequence Flow for deploy VMEnd User Rest API
SecurityCheckers
User VM Mgr
Network Mgr
Storage MgrJob
SchedulingVirtualMachine Mgr
Network Guru
Deploy VM
ACL Checks
Allocate Entity in CS
Allocate VM
Allocate NIC
Allocate Volume
Allocate IP
Schedules Deploy Job
Returns with job id, VM id
Query Job Result
Returns with job status
Sequence Flow for deploy VMJob Threads
Network Element
User VM Mgr
Network Mgr
Storage Mgr
VirtualMachine Mgr
Network Guru
Start VM
Start VM
Prepare Nics
Notify that Nic is about to be started in network
Reserve resources for Nic
Services APIServer
Resources
Start User VM
Agent Calls
Prepare Volumes
Template Mgr
Deployment
Planner
Get a Deployment Plan (Host and StoragePool)
Prepare template on Primary Storage
Agent Calls
Agent Start VM Call
Stores job result
ServerResource
• Translation layer between CloudStack commands and resource API
• May be Co-located with resource• Have no access to DB• API defined in JSON messages
DAO
• SQL generation done mostly in GenericDaoBase• Uses JPA annotations• Very little code to write for each individual DAO• Database Access Layer for Kernel• No support for more complicated features such as
fetch strategy• Welcome to use other types of ORM in other
modules but like to hear about preferred library. (Hibernate is out due to licensing issues)
Example DAO// ExampleVO.java@Entity@Table(name=“example”)public class ExampleVO { @Id @GeneratedValue(strategy= GenerationType.IDENTITY) @Column(name=“id”) long id;
@Column(name=“name”) String name;
@Column(name=“value”) String value;}
// ExampleDao.javapublic interface ExampleDao extends GenericDao<ExampleVO, Long> {}
// ExampleDaoImpl.java@Local(value=ExampleDao.class)public class ExampleDaoImpl extends GenericDaoBase<ExampleVO, Long> implements ExampleDao {
protected ExampleDaoImpl() { }}
Triggering High Availability
VM HA are triggered via the following methods:• VM Sync detects out of band VM death• Resource Management detects that a resource is
unreachable and its state can not be determined.• VM start/stop has been sent to the resource but
resource does not return• Details of how high availability is done is at
http://docs.cloudstack.org/CloudStack_Documentation/Design_Documents/CloudStack_High_Availability_-_Developer's_Guide
High Availability Future
• Moving toward using the native HA capability of the hypervisor.
• Looking to do more in the DRS area to coordinate recovery of wide spread outage.
VM Sync• Currently a sync of VM state, not entire VM• VM Sync happens between management server and hypervisor resources• Peer-to-peer sync• Hypervisor DB is considered to be the DB of truth• Two steps:
– Full Sync– Intermittent delta sync
• Establishes full sync when first connecting to the hypervisor resource• After full sync, hypervisor resource keeps track of the last sync results and only
report out of band changes on delta sync• Utilizes the most abundant resources in data center: CPU and memory• Conserve the most scarce resource: DB connections• Virtually no DB connections utilized during delta sync unless there are out of
band changes.
Storage
Zone-Level Layer 3 Switch
Pod 2 Pod N
…
Private Network
Computing Server 1
Computing Server 3
Computing Server 2
Computing Server 4
Pod-Level Layer-2 Switch
Primary Storage
Primary Storage
Pod 1
Scale-Out NFS
Primary Storage
Clus
ter 2
Clus
ter 1
Storage
• CloudStack supports two types of storage– Primary Storage: block device to the VM– WORM Storage: Secondary or Object Store for
templates, ISO, and snapshot archiving• Primary storage is high on IOPs (expensive)• Secondary storage is high on capacity (cheap)• CloudStack manages the storage between the
two to achieve maximum benefit and resiliency
Disk Offering
• Disk Offering is how disks are offered to the end user
• Disk Offering has storage tags which can be used to implementing storage tiering
• Service Offering actually contains a disk offering for the root disk
Snapshots
• Snapshots are used as backups• Taken on the primary storage and moved to
secondary storage• Full snapshots on VmWare and KVM. Need
help.• Incremental snapshots on XenServer
XenServer Snapshot
Base Copy
TemplateD1
S1D2
VM S2
Full1
S11
S12
S13
Full2
S21
S22
S23
Storage Future
• Secondary Storage multi-homed in Pod• Object Store for between zones• Direct Access to Storage for enterprise• Storage Plugins