November 12th, 20125:30 – 7:30pm
Sponsored by
“Open Source in the Cloud”
Sonian, Open Source and Sensu
November 12, 2012
Sonian’s Contributions
Fog - https://github.com/fog Elasticsearch - https://github.com/elasticsearch Openstack Swift - https://github.com/openstack/swift
Opscode Chef – https://github.com/opscode/chef/ Various Chef Tools:
https://github.com/portertech/chef-metrics https://github.com/portertech/chef-journalist https://github.com/portertech/recognizer https://github.com/portertech/chef-irc-snitch
Home Built and Released
SCLI (Smart Cloud Command Line interface) https://github.com/sonian/scli (MIT)
Amazon-Pricing (Pricing Gem) https://github.com/sonian/amazon-pricing (Ruby)
ElasticSearch Jetty Plugin https://github.com/sonian/elasticsearch-jetty (Apache
2)
Sensu – Monitoring Framework https://github.com/sensu (MIT)
Sensu – “The Monitoring Router” Monitoring Framework – Built for the cloud (Dynamic Environments)
Ruby (EventMachine, Sinatra, AMQP), RabbitMQ, Redis Messaging oriented architecture. Messages are JSON
objects. (Pub/Sub) Ability to re-use existing Nagios plugins Plugins and handlers (think notifications) can be
written in any language Designed with modern configuration management
systems such as Chef or Puppet in mind Lightweight, less than 1200 lines of code
Why We Built It
Highly Elastic Infrastructure Nodes are created (Spot Nodes) Bootstrapped (With Chef) Take and process work Terminated (when prices increase)
All before they are discovered and monitored by Nagios
Nagios is: Difficult to Extend Can not discover new services on its own Generally Unpleasant
Keep It Simple™
• The Idea:• Schedule the execution
of remote checks• Collect their Results
• “Checks” are:• Is the server up?• How hard is it working?
• Tied into Modern CM• Chef• Puppet
• Message Oriented Middleware
• RabbitMQ• Securely Routing
Checks/Results• Redis: Fast In-Mem K/V
Store
Open Source = Community
• Early Development – Recruit Community Experts• Help Test – Drive Early Roadmap• Develop Puppet and Chef modules
• Release Day (Nov 1st 2011)• Make Sensu Github Repo Public• Open IRC channel on Freenode (#sensu)• Blog posting and Twitter for marketing
• Community, Community, Community• Adoption – Documentation• “Omnibus” Style Packaging for Quick Deployments
Contact Pete Cheslock
Director of Technical & Cloud Operations @ Sonian
@petecheslock
http://about.me/petecheslock
We’re Hiring! Please contact Sonian’s VP of Product Development
Glenn Snyder [email protected]
Global Marketing
Outline
• Level Set on Current Cloud Trends, pain points, and markets
• Open Source Trends• OpenStack Introduction
–What it is–Historical Evolution–Use Cases–Conceptual Architecture–Customer Adoption–Cloud Taxonomy
11
Global Marketing
Level Set On Cloud trends
12
Global Marketing
Cloud is about Services, Not Systems: Consumer Market Driving TrendsCloud is about Services, Not Systems: Consumer Market Driving Trends
13
24/7 CLOUD SERVICES
Anywhere, Anytime, Internet Access
Interconnected mobile devices
Global Marketing
Enterprises: Same Cloud Services Goals But Different Market Segments and Pain Points
14
Mega Datacenters– Homogeneous– Dense/Low Cost
Small Site (SMB/Branch)– Low Cost– Ease of deployment– Ease of management
Enterprise Datacenter– Heterogeneous– Legacy / Complex– Consolidating Quickly
Mega Datacenter
Enterprise Datacenter
Small Site (SMB/Branch)
• Burdened by “legacy” infrastructure and applications
• Too complex to manage• Expensive to implement
and maintain• Takes too long to realize
value• Not easy to innovate• Security, compliance,
and privacy concerns
• Source: IDC
Global Marketing
Virtual Machines Already Outstripping Physical Machines
“2012: RATE(of VMs launched per sec 6/sec) > RATE (at which babies are born in the US per sec 8/sec) (from #VMW) #cloud”
Global Marketing
Extensive Use of
SAN
Complex Networks
Dense Blades
>$2000 / VM
Specialized Skillset
Fixed Capacity
But what about storage and networks?
Global Marketing
Better resource utilization for each server
Virtualization 1.0: Not up to the challengeVirtualization 1.0: Not up to the challenge
HOST 1 HOST 2 HOST 3 HOST 4, ETC.
VM
Hypervisor(VMWare ESX, Citrix XEN Server, KVM, Etc.)
Hardware abstraction for each server
2. Cloud Data Center 3. Cloud Federation1. Server Virtualization1. Server Virtualization
Automation & Efficiency
Global Marketing
+
How do you empower people to self-service?How do you empower people to self-service?
USERS ADMINS
How do you make your apps cloud aware?How do you make your apps cloud aware?
Where should you provision new VMs?Where should you provision new VMs? How do you keep track of it all?How do you keep track of it all?
But questions arise as the environment grows...“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
APPS
2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization
Automation & Efficiency
Global Marketing
USERS ADMINS
But questions arise as the environment grows...“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
A Cloud Management Layer Is MissingA Cloud Management Layer Is Missing
APPS
2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization
Automation & Efficiency
Global Marketing
APPS
What is needed is a cloud Operating System that adds automation and control at scale
Creates Pools of Resources Automates The Network
USERS ADMINS
CLOUD OPERATING SYSTEMCLOUD OPERATING SYSTEM
Connects to apps via APIs Self-service for users
2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization
Automation & Efficiency
Global Marketing
Compute Pool Network Pool Load Balancing Pool
Image Service PoolStorage Pool
Types of pools managed by the Cloud O.S.COMPUTE, NETWORK, & STORAGE
2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization
Automation & Efficiency
Global Marketing
Assembly Line IT
Global Marketing
Robotics Factory IT
Global Marketing
Open Source, OpenStack Evolution and Trends
24
Global Marketing
Innovation + Open Source = Virtuous Cycle
• Open source is leading, not following, in important areas including cloud, big data, mobile apps and enterprise mobility.
• More than 50 percent of software acquired in the next five years will be open source software.
• Innovation, flexibility, cost, quality
of open source are some of the top reasons that make it attractive for use
• Companies most likely to be impacted by OSS have these characteristics: Business drivers to invest in IT, software development is an essential strategic process, technology centric
• When asked about revenue generating strategies likely to create value for vendors, 52 percent of respondents said an annual, repeatable support and service agreement was the most likely value strategy
• http://northbridge.com/2012-open-source-survey
Open Source Innovation
600,000+ OSS projects
100+ billion lines of code
10 million person-years of work
Source: Blackduck
Global Marketing
3. Cloud Federation3. Cloud Federation
Automation & Efficiency
OpenStack is the Open Source Software Powering Public and Private Clouds
Public Cloud:OpenStack powers someof the worlds largest publiccloud deployments.
2. Cloud Data Center2. Cloud Data Center1. Server Virtualization1. Server Virtualization
Private Cloud:Run OpenStack softwarein your owndata centers
Global Marketing
• What is it: An open source cloud operating system– 30,000 lines of code to 600,000 in under 18
months• Who’s building it: A worldwide community of
developers– 600+ developers, 250+ contributed in the last
12 months– 6000+ individual members in 87 countries
• Who is govering it: The OpenStack Foundation, backed by AT&T, Canonical, Cisco, ClearPath, Cloudscaling, Dell, DreamHost, HP, IBM, ITRI, Mirantis, Morphlabs, Nebula, NetApp, Piston, Rackspace, Red Hat, SUSE, and Yahoo! (so far)
Global Marketing
Wide-Ranging Community Support….
Confidential28 04/10/23
Global Marketing
OpenStack Timeline
2011
Feb 2011:Bexar
Release
Apr 2011:Cactus Release
Sep 2011:Diablo
Release
Apr 2012:Design Summit
Austin
Formation
Austin
Formation
Bexar
First Public Code
Bexar
First Public Code
Cactus
Community DevelopmentForming
WorkingPrototypes
Cactus
Community DevelopmentForming
WorkingPrototypes
Essex
“Production Ready”
Stable Foundation
Included in Ubuntu 12.04
Incubated: Network & Block Storage
Essex
“Production Ready”
Stable Foundation
Included in Ubuntu 12.04
Incubated: Network & Block Storage
2012
Nov 2010 Dec Feb Apr Jun Aug Oct Dec Feb Apr
Oct 2011:Design Summit
Mar 2012:Essex
Release
Nov 2010:
Austin ReleaseOct 2010:
Design Summit
Apr 2011:Design Summit
Diablo
Workable Foundation
Exposes GapsSolidify Community
Loses VMware & HyperV
Diablo
Workable Foundation
Exposes GapsSolidify Community
Loses VMware & HyperV
Folsom
“Platform for Innovation”
Core Platform for Innovation
Network as a ServiceBlock Storage
Public AdoptoinMultiple Scale Deployments
Folsom
“Platform for Innovation”
Core Platform for Innovation
Network as a ServiceBlock Storage
Public AdoptoinMultiple Scale Deployments
Jun Aug
Oct 2012:FolsomRelease
Global Marketing
• Addresses Real Market Pains– Limits costly software licenses– Limits lock-in by vendors (VMware) & by
providers (Amazon)– Allows for massive scalability– Extensible hypervisor support (Xen, KVM,
Hyper-V, etc.)– Offers standard APIs enabling growing cloud
ecosystem
OpenStack is commoditizing the IaaS market from single provider (Amazon)
to many small copy cats (startups).
Lock-in & Licenses
Open APIs& Support
ProvidesReduce
s
OpenStack Value Proposition
Global Marketing
Popular OpenStack Use Cases
• Service providers offering an IaaS compute platform
• IT departments provisioning compute resources to teams and projects
• Processing big data with tools like Hadoop
• Scaling compute up and down to meet demand for web resources and applications
31
Global Marketing
OpenStack Conceptual Architecture
Confidential32 04/10/23
Global Marketing
There is a broad adoption of OpenStack across many markets. The most common markets we have seen so far have been:
UniversitiesHealthcareCDN providersHosting providersStart-upsSaaS companiesGovernment
Confidential33
Global Marketing
Vision for Complete OpenStack Solution
Global Marketing
Complete Cloud Taxonomy
Admin SoftwareWeb Services & APIs
Customer Management
Entitlement, rights Billing
Self Service Portal
Metering
Infrastructure Software
Abstraction Software
Physical
Software as a Service
IT as a Service
Platform as a Service
Infrastructure as a Service
Everything as a Service
HVAC Power Facility
Network
Environmentals
Compute Switch Storage
Operating System Virtualization
Application Run-Time
Virtualization
Data Store
Monitoring
Intelligent Resource Manager
Workload Lifecycle Management
Platform Provisioning
Ser Gov/Workflow Automation
Orchestration
Info
rmati
on
Serv
ice M
an
ag
em
en
tLeg
acy M
an
ag
em
en
t
Fir
ew
al
l
Rep
ort
ing
An
aly
tics
IPS
Secu
rit
yLD
AP
/A
DS
SO
Overarching Systems
Global Marketing
Disruption
Global Marketing
Discussion