Date post: | 16-Apr-2017 |
Category: |
Technology |
Upload: | mesosphere-inc |
View: | 646 times |
Download: | 0 times |
Re-Platforming All the Things
© 2015 Mesosphere, Inc.
Who in the world are you?
Thomas Rampelberg
Product Manager
@grampelberg
2
© 2015 Mesosphere, Inc.
Who do you work for?
3
© 2015 Mesosphere, Inc.
Interested in trying it out now?
• Mesosphere DCOS• Apache Mesos + Marathon• https://mesosphere.com/amazon/
4
© 2015 Mesosphere, Inc.
• The datacenter of yesterday• The datacenter of today• The datacenter of tomorrow
Agenda
5
© 2015 Mesosphere, Inc. 6
Yesterday
© 2015 Mesosphere, Inc.
History of the World
7
The Mainframe
Linux + x86
Virtual Machines / Cloud
Containers
© 2015 Mesosphere, Inc.
Mainframes
• Applications - 1 service• Hardware - 1 box• Redundancy - Systems Architecture• Reboot - Never• Services - 100s/”host”
8
© 2015 Mesosphere, Inc.
• Applications - 2(ish) services• Hardware - 10s of boxes• Redundancy - Multiple boxes• Reboot - Occasionally• Services - 10s/”host”
Linux + x86 - Commoditization
9
© 2015 Mesosphere, Inc.
Virtual Machines / Cloud
• Applications - 3+ service• Hardware - 100s of “hosts”• Redundancy - Hardware + Software• Reboot - Occasionally• Services - A couple/”host”
10
© 2015 Mesosphere, Inc.
• Applications - Microservices• Hardware - 1000s of containers• Redundancy - Multiple boxes• Reboot - Often• Services - 1 per container
Containers
11
© 2015 Mesosphere, Inc. 12
Today
© 2015 Mesosphere, Inc.
Pets
• They’re Individuals• You have to clean up• They fail eventually
13
© 2015 Mesosphere, Inc.
• Treated as a herd• Cow pies happen• They fail eventually
Cattle
14
© 2015 Mesosphere, Inc.
How are these different?
• 144 Cores• 1.15 TB Memory
15
• 144 Cores• 1.15 TB Memory
© 2015 Mesosphere, Inc.
“Sixty-four cores or 128 cores in a single computer looks a lot like 64 or 128
machines in a datcenter” - Benjamin Hindman
16
© 2015 Mesosphere, Inc.
One Big Computer
• Manage Resources Globally• Manage Failures Globally• Scale Across Hosts
17
© 2015 Mesosphere, Inc.
• Scalability• Multi-tennancy• Efficiency• Reliability & availability• On-demand resources
What do you get?
18
© 2015 Mesosphere, Inc.
Who’s doing it?
Closed Source
• Google Borg• Google Omega
19
© 2015 Mesosphere, Inc.
Who’s doing it?
20
Open Source
• Apache Mesos• Apache Hadoop (YARN)• Kubernetes
© 2015 Mesosphere, Inc.
Apache Mesos
21
© 2015 Mesosphere, Inc.
Overview
• Borg is awesome, let’s do something better
• Came out of UC Berkeley's AMP Lab• First paper in 2009• Running at Twitter and Apple
22
© 2015 Mesosphere, Inc.
• Separate resources from placement• Focus on running batch jobs and
services• All about the scheduler
What’s Different?
23
© 2015 Mesosphere, Inc.
Apache Hadoop (YARN)
24
© 2015 Mesosphere, Inc.
Overview
• Hadoop needed a better scheduler• YARN became the foundation for
Hadoop 2.0• Paper came out in 2013• Running at Yahoo and LinkedIn
25
© 2015 Mesosphere, Inc.
• Centralize the resource and placement decisions
• Optimize for somewhat long batch jobs (eg. MR)
• Bake the storage layer into the solution
What’s Different?
26
© 2015 Mesosphere, Inc.
Kubernetes
27
© 2015 Mesosphere, Inc.
Overview
• Google took everything they learned from Borg and Omega
• Kubernetes was open source in 2014, just went 1.0 last month
• Running in production ….. ?
28
© 2015 Mesosphere, Inc.
• Containers first, only containers• Pods are a first class citizen• Labels, labels everywhere
What’s Different?
29
© 2015 Mesosphere, Inc. 30
Tomorrow
© 2015 Mesosphere, Inc.
Stateful Applications
• Persistent Volumes• Durable Pods• Distributed File Systems
31
© 2015 Mesosphere, Inc.
• Networking virtualization• Flannel• Weave• Calico
• Dynamic load balancing• IPv6
Service Discovery
32
© 2015 Mesosphere, Inc.
Security
• Not everyone is a small startup• Compliance is important• Long tail of process• Containers are still early• Schedule policy, automatically
33
© 2015 Mesosphere, Inc.
• Sizing is hard• Not all processes need guarantees• Let the scheduler sort it out
Oversubscription
34
© 2015 Mesosphere, Inc. 35
Next Steps
© 2015 Mesosphere, Inc.
Getting Started
• Apache Mesos - http://mesosphere.com/product
• Apache Hadoop (YARN) - http://bit.ly/1K13E4S
• Kubernetes - http://kubernetes.io/gettingstarted/
36
© 2015 Mesosphere, Inc.
• http://chat.mesosphere.com/• http://mesos.apache.org/community/• https://wiki.apache.
org/hadoop/HowToContribute• http://kubernetes.io/community/
Get Involved!
37
© 2015 Mesosphere, Inc.
Get Awesome
38