Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
6th Slide Set Cloud Computing
Prof. Dr. Christian Baun
Frankfurt University of Applied Sciences(1971–2014: Fachhochschule Frankfurt am Main)Faculty of Computer Science and Engineering
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 1/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Agenda for Today
Solutions for running private cloud infrastructure servicesFocus: Eucalyptus and OpenStack
Solutions for running private platform servicesFocus: AppScale
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 2/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Solutions for running Private Cloud Infrastructure Services
Several free solutions exist run infrastructure services
abiCloud (Abiquo) http://www.abiquo.comCloudStack (Citrix) http://cloudstack.apache.orgEnomaly ECP http://src.enomaly.comEucalyptus http://open.eucalyptus.comNimbus http://www.nimbusproject.orgOpenECP http://openecp.sourceforge.netOpenNebula http://www.opennebula.orgOpenStack http://www.openstack.orgTashii (Intel) http://www.pittsburgh.intel-research.net/projects/tashi/
These solutions are used mainly for the construction of private cloudsSome solutions can also be used for the construction of public cloudservices
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 3/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Project Status of the Solutions
abiCloud (Abiquo) ???Apache CloudStack https://github.com/apache/cloudstackEnomaly ECP d (The company discontinued its former open source strategy)Eucalyptus (d) https://github.com/eucalyptus/eucalyptusNimbus (d) https://github.com/nimbusproject/nimbusOpenECP d (Fork of Enomaly ECP. The development is stopped)OpenNebula https://github.com/OpenNebula/oneOpenStack https://github.com/openstackTashi (Intel) d (Intel is strongly committed with OpenStack)
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 4/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Eucalyptus
EUCALYPTUS – Elastic Utility Computing Architecture for LinkingYour Programs To Useful SystemsAllows execution and control of virtual instances (Xen or KVM) ondifferent physical resourcesDeveloped at UC Santa Barbara
Further development by Eucalyptus Systems, Inc.Interface compatible to AWS
EC2 + EBS + ELB + AutoScaling and S3Use of poplar AWS-compatible tools is possible:
e.g.: S3 Curl, Elasticfox, s3cmd,. . .Free software: GPLv3 (until 11/2017). Now BSD
In 2008 + 2009, Eucalytus was a major step forward in establishing an API standard for cloudinfrastructure services
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 5/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Eucalyptus Services
Eucalyptus consists of several UNIXservices
Cloud Controller (CLC)Cluster Controller (CC)Node Controller (NC)WalrusStorage Controller (SC)
The services communicate via webservices (SOAP+REST)Eucalyptus infrastructures consistof one or more sites
Redundant operation of the services CLC, CC,Storage Controller and Walrus became afeature with Eucalyptus v3.0 in 2011This feature was removed with v4.2 in 2015
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 6/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Node Controller (NC)
Runs on every physical node, whereinstances are planned to runControls the KVM hypervisor
Xen is not supported any longer since v4.0VMware ESX(i) is not supported any longersince v4.1
Each NC transmits informationabout the utilization of their ownresources to the CC of the site
Number of virtual processorsFree memoryFree storage
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 7/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Cluster Controller (CC)
Exactly a single CC per site isrequiredControls the distribution of thevirtual machines to the NCsCollects free resource informationfrom the NCsIn small infrastructures CLC andCC usually run on the samephysical serverIn each site, the NCs communicatewith the CC via a virtual network(VLAN)
The VLAN ensures that allinstances within a site share thesame subnet
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 8/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Cloud Controller (CLC)
Exactly a single CLC perEucalyptus infrastructure is requiredActs as a meta-scheduler in thecloud infrastructureCollects resource information fromthe CCsRuns per default on the samephysical server as the storageservices Walrus and StorageController
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 9/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Walrus
Storage service, which implementsthe S3 REST APIEucalyptus stores here the imagesUsually runs on the physical serverwhich hosts the CLC/CC
Can be outsourced from the CLCsince v1.6
Walrus is not a distributing serviceOperates only in single-nodemode
In order to improve the read/writeperformance of the object-basedstorage, Walrus can be replaced bya Riak Cloud Storage (Cluster)
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 10/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Storage Controller
Storage service, which implementsthe EBS APIIn infrastructures with only a singlesite, the Storage Controller usuallyruns on the physical server whichhosts the CLC/CC
Can be outsourced from the CLCsince v1.6
If the infrastructure containsmultiple sites, each site has its ownstorage controller
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 11/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Launch of an Instance in Eucalyptus (1/2)
1 A user or applications, which triesto start instances, provides the CLCthese parameters:
ImageIinstance typeNumber of instances
2 CLC selects a CC with enough freeresources in its cluster
3 CC selects in the local cluster one(or more) NC(s), with enough freeresources and commands the startof the instance(s)
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 12/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Launch of an Instance in Eucalyptus (2/2)
4 If the required image is notavailable on the NC, the NCrequests the image from the CLC
5 CLC transmits the image fromWalrus via an encryptedtransmission via Secure Copy(SCP) to the NC
6 The transmission duration forimages from Walrus to the NCsdepends of:
Network technology usedNumber of required transmissionsSize of the images
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 13/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Some Facts about Eucalyptus
The Installation of Eucalyptus is simple when CentOS or Red HatEnterprise Linux are used
and it is horror when other Linux distributions are usedStable operation of an Eucalyptus infrastructure is possible
If administrators are willing to invest some time. . .Single services need to reboot from time to timeHelp of the developers is not always helpfulCommercial support is (was?!) availableCommercial interests seem(ed) to be contrary to community conceptsometimes
Extensions and modifications in the source code are difficultSource code of services appears obscureNo assistance from the developers can be expected
Unclear futureHP aquired Eucalyptus systems in 9/20142015: Eucalpytus became a part of HPE, which separated from HP2017: Eucalpytus became a part of DXC Technology =⇒ d (?!)
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 14/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
A new Hope for Eucalpytus?
DXC stopped developing theproduct in late 2017AppScale Systems forked the codein 2018 and started supporting theproducthttps://www.eucalyptus.cloud
Latest version: 4.4.5 fromDecember 2018https://github.com/corymbia/eucalyptus/
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 15/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Eucalyptus Installation (the simple way on a single node)
Check the installation tutorialhttps://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#shared/install_section.html
Create a virtual machine with CentOS 7.3 minimalExecute:bash <(curl -Ls https://eucalyptus.cloud/install)
The script will ask a few questions (e.g. about spare IP addresses)Hope the best
A faststart iso image „Cloud in a Box“ existed until 2015/2016The last revision came with Eucalyptus 3.4.2 and CentOS 6It was the most simple way to install an Eucalyptus IaaS on a single node or a cluster modeIt is not available any more
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 16/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Review of NASA regarding Eucalyptus
Source: http://www.theregister.co.uk/2010/07/20/why_nasa_is_dropping_eucalyptus_from_its_nebula_cloud/
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 17/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
2011: Ubuntu switches from Eucalpytus to OpenStack
Source: http://talkincloud.com/ubuntu-cloud-openstack-wins-eucalyptus-loses
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 18/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
OpenStack Image Source: OpenStack
Initiated by NASA and Rackspace CloudSupported by AMD, Dell, IBM, Intel, Red Hat, SuSE, Yahoo and more
Free software (Apache License v2.0)Contains several services which communicate via REST
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 19/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
OpenStack – Services (1/4)
Compute (Nova)Infrastructure serviceImplements the EC2APIHighly scalable (up totens of thousands ofnodes)
Object Storage (Swift)Redundant, highly scalable (petabyte range), object-based storage serviceObjects are stored on multiple hardwareAutomatic replication when nodes fail or are addedImplements the S3 API
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 20/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
OpenStack – Services (2/4)
Image Service (Glance)Service for the search,register and request ofimagesSupported imageformats: Raw, AMI,VHD (Hyper-V), VDI(VirtualBox), qcow2(Qemu/KVM), VMDKand OVF (VMWare)
Block Storage (Cinder)Storage service for persistent block-based storage devicesVirtual storage devices can be created, erased, attached to and detachedfrom instancesImplements the EBS API
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 21/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
OpenStack – Services (3/4)
Networking (Neutron)Service for managingIP addresses anddistributing them toinstancesAdministrators specify,if all instances areconnected to the samenetwork, or if they areseparated from eachother via VLAN
Dashboard (Horizon)Provides a graphical web-interface for administrators and users
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 22/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
OpenStack – Services (4/4)
Identity Service (Keystone)Central directory of users for the other OpenStack servicesProvides user authenticationCan interact (query) existing user directory services (e.g. LDAP)
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 23/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Infrastructure Services and their Compatibility to the AWS
AWS APIs implementedProject/Solution EC2 S3 EBS (EC2)abiCloud (Abiquo) partly — —CloudStack (Citrix) partly — —Enomaly ECP — — —Eucalyptus partly partly (Walrus) partly (SC)Nimbus partly partly (Cumulus) —OpenECP — — —OpenNebula partly — partlyOpenStack partly (Nova) partly (Swift) partly (Cinder)Tashi (Intel) — — —
Many free private cloud solutions exist, which implement anobject-based storage service and provide the S3 API
Ceph-RGW, Fake S3, Minio, Riak CS, S3ninja, S3rver, Scality S3Server,. . .
Freie Objektspeichersoftware mit S3-API, Christian Baun. iX 9/2017, P.76-79https://www.heise.de/ix/heft/Eimerweise-3807215.html
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 24/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
AppScale
Free reimplementation of the Google App EngineAllows the operation of a GAE-compatible platform service
Developed at UC Santa Barbara since 2008Free software (Apache 2.0 license)GAE-compatible applications can be developed, executed and testedwith AppScale inside a. . .
public cloud with EC2, Microsoft Azure, Google Compute Engine orAlibaba Cloudprivate cloud with Eucalyptus, OpenStack and CloudStack
Can also run directly on Docker, Xen, KVM and VirtualBoxActively developedhttps://github.com/AppScale/appscale
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 25/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
AppScale Implementation (1/3)
AppScale emulates Google’s infrastructure services by using freesoftwareThe Application server with its APIs is implemented by using the AppEngine SDK (which is free software)
The SDK contains a resource-saving development server, thatimplements the core functionality and the APIs of the PaaSBut this development server does not scale per default!
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 26/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
AppScale Implementation (2/3)
The web server functionality is implemented with nginxThe load balancing functionality is implemented by HAProxyA Datastore implements the distributed NoSQL db Apache CassandraThe coordination service ZooKeeper synchronizes all services
Apache ZooKeeper is a distributed configuration service, synchronizationservice, and naming registry for large distributed systems
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 27/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
AppScale Implementation (3/3)
The messaging functionality provides the free XMPP server ejabberdThe memcache storage is emulated with the cache server MemcachedThe execution of regularly scheduled tasks that operate at defined timesor regular intervals is done by cronThe image manipulation functios provides the Python Imaging LibraryMessage queues implements AppScale with RabbitMQ
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 28/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
AppScale Installation
As first step, the AppScale Tools need to be deployedThis can be done via pip, which is the package manager for pythonpackages
$ sudo pip install appscale-tools
Now, the command line tool appscale is availableNext, the AppScale PaaS service need to be deployed
This can be done either inside. . .a VM inside a public cloud service offeringa local VM or containerdirectly inside the local host operating system
https://www.appscale.com/try-appscale
≥ 4GB of main memory are required because of the Cassandra DB
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 29/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Deployment of AppScale inside a local VirtualBox VM
$ mkdir -p ~/appscale$ cd ~/appscale$ vagrant init appscale/releases$ curl -o Vagrantfile https://s3.amazonaws.com/appscale_CDN/files/Vagrantfile_template$ vagrant up
More information:https://github.com/AppScale/appscale/wiki/AppScale-on-VirtualBox
The installation with Docker is even more simple:docker run -i -t appscale/appscale:latest /bin/bash
AppScale als Alternative zu Googles App Engine, Christian Baun. iX 12/2016, P.72-75
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 30/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Working with AppScaleCreate an AppScalefile$ appscale init clusterModify your AppScalefile (this is for a single machine configuration)ips_layout:
master : <IP-Node-1>appengine: <IP-Node-1>database: <IP-Node-1>zookeeper: <IP-Node-1>
Start AppScale$ appscale upDeploy a web application$ appscale deploy guestbook.tar.gzThe application is online now via port 8080 (HTTP) and 4380 (HTTPs)Shut down AppScale$ appscale downErase the web application and all data$ appscale clean
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 31/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Create an AppScale Cluster
Use 4 or 8 nodesWhen using 4 nodes, assign each node one of these roles:master, appengine, database and zookeeper
Drawback: No redundant storage or servicesWhen using 8 nodes, only the master is the Single Point of Failureips_layout:master : <IP-Node-1>appengine:- <IP-Node-2>- <IP-Node-3>database:- <IP-Node-4>- <IP-Node-5>zookeeper:- <IP-Node-6>- <IP-Node-7>- <IP-Node-8>
More information:https://github.com/AppScale/appscale/wiki/Supported-Deployments
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 32/33
Infrastructure Services (Eucalyptus + OpenStack) Platform Services (AppScale)
Summary
The discussed solutions for running private cloud infrastructure andplatform services all have advantages and drawbacksDrawbacks:
Stability and functionality are usually worse in contrast to public cloudservice offerings
Benefits:Independence from the behavior of public cloud services providers
Examples: business model changes (usage fee increases), functionality ofthe services change, service quality changesPrecondition: APIs must be compatible with public cloud services offerings
Free software =⇒ high level of flexibilityPotential way to build hybrid clouds
Conclusion:Even if the stability and functionality are lower, private cloud services canprotect against unwanted changes
Prof. Dr. Christian Baun – 6th Slide Set Cloud Computing – Frankfurt University of Applied Sciences – WS1819 33/33