State of the art from C. One - 1

State of the art from Compatible OneJean Parpaillon

State of the art from Compatible OneJean Parpaillon

State of the art from C. One - 2

The ProjectsThe Projects

State of the art from C. One - 3

● CompatibleOne● « a meta-model derived from the concepts of Grid Computing of contracts for the supply of resources »

● « a free/open source reference Cloudware stack that can be freely installed on any server to supply, manage, control and exchange resources »

State of the art from C. One - 4

● Sub-projects● Infrastructure as a Service● Platform as a Service● Security, QoS and Management

State of the art from C. One - 5

● Aeolus● « developing theory and tools to automate deployment, reconfiguration, and upgrades of variable sized, non-homogeneous machine pools »

State of the art from C. One - 6

DirectionsDirections

State of the art from C. One - 7

● Requires● Cloud description model and language● Cloud modification request language● Low level deployment language

State 1 State 2

State of the art from C. One - 8

Cloud description (1)

● Describing (virtual) hardware resources● Grid models

– Computing– Memory– Storage– Networking

● IaaS models– Grid +– Elasticity– Virtualization

State of the art from C. One - 9

Cloud description (2)

● Software resources● Distribution packaging systems

– deb– Rpm– Etc.

● Software configurations● Configuration management tools

– Puppet, Chef, cfEngine

State of the art from C. One - 10

Cloud description (3)

● Describing relations in distributed system● Not found...

State of the art from C. One - 11

Modification request description (1)

● Modification of the system...● Virtual hardware● Packages● services

– cf cloud description

State of the art from C. One - 12

Modification request description (2)

● ...with non-functional requirements● Costs, QoS, location, etc.● Need for limiting request 

– “I want my new cloud provider CEO to wear nice socks”● Look at existing:

– Ontologies: things and their relations– Taxonomies: hierarchical structures

State of the art from C. One - 13

Low-level deployment language

● A minimum of abstraction:● Package management● Service management● Configuration management

● Dependancy handling● Distributed deployment of configuration

State of the art from C. One - 14

Cloud modelsCloud models

State of the art from C. One - 15

Cloud model : original stack

IaaS : Infrastructure (virtual) CPU, memory, network, etc.

PaaS : Platform Comprehensive API stacks

SaaS : Application Turnkey applications

State of the art from C. One - 16

Cloud model : refined stack

Hardware Concrete hardware : CPU, harddisk, etc.

Virtualization Virtual hardware : Xen box, KVM box, etc.

Cloud Virtual datacenter : OpenStack, EC2, etc.

Accounting Accountable resources

Scheduling Resources requests

Platform Comprehensive APIs

Appliance (Distributed) application

State of the art from C. One - 17

Cloud model : « everything is a process »

● « SlapOS: a Multi-purpose Distributed Cloud Operating System Based on an ERP Billing Model » http://www.slapos.org/slapos-Wiki.Home/slapos-Smets.Cerin.Courteaud.IEEECloudPerf2011

● Inspired from grid systems

Application

Resources description

SlapOS Process

State of the art from C. One - 18

Cloud resources management systemsCloud resources management systems

State of the art from C. One - 19

Cloud engines

● IaaS oriented● Deploying virtual machine images on allocated resources

● Storing and retrieving virtual images● Migrate vm from one hardware to another● One or several API: OCCI, EC2, native● No deal with packages, services, etc.

State of the art from C. One - 20

OpenNebula – short story

● Started in 2005● First public release 2008, Apache License● Mostly from research projects● Recent commercial support (C12G)● Large existing deployments:

● CERN (http://blog.opennebula.org/?p=620)

State of the art from C. One - 21

OpenNebula - features

● Manages all aspects of infrastructure● Users, images, hypervisors, network, storage● Scheduling: 

– Basic: CPU load, memory load, locality– Advanced: Haizea (http://haizea.cs.uchicago.edu/)

● Support public clouds API: OCCI, EC2● Included monitoring

State of the art from C. One - 22

OpenStack – short story

● Started by 2010● FOSS alternative to Eucalyptus● Strong  community● Sponsored by the NASA● Aiming at scalability

●  1 million hosts, 60 millions guests● 100 PB / cluster● 100 000 requests/sec

State of the art from C. One - 23

OpenStack - features

● Manages● Role Based Access Control● Hypervisors● Integrated storage (SWIFT)● Images repository and retrieval (GLANCE)

→ Promising but younger than OpenNebula

State of the art from C. One - 24

SlapOS (1)

● Started late 2010● Developed by Nexedi● Provides

● Infrastructure: kvm, libcloud, etc.● Platform: Java, Python, *SQL servers, etc.● SaaS: wordpress, xwiki, ERP5

State of the art from C. One - 25

SlapOS (2)

● Relies on: ● linux + libc● Buildout

● Efficiency: no virtualization● Application → /opt/<application>● Process is managed through supervisord

State of the art from C. One - 26

Resources descriptions models and languages

Resources descriptions models and languages

State of the art from C. One - 27

OCCI – short story

● “The Open Cloud Computing Interface (OCCI) is a RESTful Protocol and API for all kinds of Management tasks”

● Started March 2009● Today 250 members from industry, academia and research

State of the art from C. One - 28

OCCI - features

● Aiming at:● Interoperability● Integration with IaaS, PaaS, SaaS● Extensibility

● Parts implemented in:● OpenStack, OpenNebula, etc.● http://occi-wg.org/community/implementations/

● No reference implementation● Current specifications mostly concerns IaaS

State of the art from C. One - 29

Cloud resources taxonomy

● Proposed by SlapOS project● http://www.slapos.org/P-VIFIB-Default.Category.Configuration/view

● Covers● Infrastructure (CPU, memory, storage, etc,)● Countries, skills, functions, etc.

● To be completed in CompatibleOne project

State of the art from C. One - 30

SNMP

● Mature taxonomy (first RFC: 1988)● IETF + IANA = 317 MIB modules

● Describes● Network systems (legacy use)● Complex computing systems

– Web servers– Software and hardware resources

● Still evolving: LIBVIRT-MIB

State of the art from C. One - 31

Configuration management toolsConfiguration management tools

State of the art from C. One - 32

Classification

● Configuration files abstraction:● 'Sed ...' vs 'set(file, section, key, value)' ?

● Distribution abstraction:● Packages manager, services start/stop

→ Declarative vs imperative language● Dependancy handling

● 'make world' vs 'make earth wind fire'

● Distributed configuration

State of the art from C. One - 33

Augeas

● Not a configuration management tool :)● Abstract configuration storage:

● Config → tree of key=value

● API: C, CLI, Python, Ruby, Haskell, Java, …● Repository of parsers : xorg, sshd, mysql, ...

State of the art from C. One - 34

CfEngine (1) 

● Grandfather of config mgmt tools (1993)● Version 3.0.2, June 2010● Based on “promises”:

● Describe desired state to reach● Domain Specific Language

● Low CPU and memory footprint

State of the art from C. One - 35

CfEngine (2)

● Decentralized architecture● Agents pull config and try to reach promises● Asynchronous execution

● Graph-ordering model

State of the art from C. One - 36

Puppet (1)

● First release 2005● Written in Ruby● Large community and commercial support● Architecture:

● Declarative DSL● Client-Server model● REST API

State of the art from C. One - 37

Puppet (2)

● Integrates:● Modules repository (retrieving and submission)● Augeas (config file abstraction)● Resources abstraction: user, package, service,...

● Performances / cfEngine● Dependancy handling● Graph-ordering model

State of the art from C. One - 38

Chef (1)

● Inspired by Puppet● First release 2009● Written in Ruby● Architecture:

● Client-server● Rely on HTTP server

State of the art from C. One - 39

Chef (2)

● Convention based● File names, dir names

● Recipes in pure Ruby● Already large repository of recipes● Procedural ordering model

State of the art from C. One - 40

bcfg2

● First release in 2004● Written in Python● Declarative configuration description in XML● Specificity: reporting system

● Check the configuration correctness● Report differences  between client state and config● Dry-run mode

State of the art from C. One - 41

Buildout

● Build system● Written in Python● Designed for Python but not only

● cf SlapOS

State of the art from C. One - 42

GNU make

● Build system● Well-known● Lightweight● Relies on external tools for abstraction

● Packages, users, services, etc.

State of the art from C. One - 43

ConclusionConclusion

State of the art from C. One - 44

Cloud modeling

● Well known objects: ● virtual CPUs, memory, packages...

● ...and some other not:● User requests● Services

State of the art from C. One - 45

Thank you !Thank you !