Date post: | 05-Jan-2017 |
Category: |
Software |
Upload: | chris-aaron-gaun |
View: | 222 times |
Download: | 6 times |
©2016 Apigee Corp. All Rights Reserved.
Kubernetes: Managing Microservices at Scale
APPRENDA.COM
Isaac “Ike” Arias
Sr. Director
Worldwide Solutions Engineering
Apprenda
@iketweets
APPRENDA.COM
A (short) story about containers...
APPRENDA.COM
chroot(1979)
BSD Jails(2000)
Solaris Zones(2004)
Linux CGroups(2006)
LXC(2008)
Docker(2013)
rkt(2014)
LXD(2014)
OS-level Virtualization Is Not New
1980 1990 2000 2010
APPRENDA.COM
Process
Process
Process
Process
Process
Process
Net I/O
CPUM
em
Mount
PIDNet
IPCUT
SUs
er
Process
OS Containers = Isolation + Resource Management
APPRENDA.COM
Process
Linux Container Technologies
Namespaces
CGroups
AppA
rmor SELinux
APPRENDA.COM
Why Containers?PERFORMANCE • SIZE • REPEATABILITY • ISOLATION • QUALITY OF SERVICE • ACCOUNTING • VISIBILITY • PORTABILITY
Most agile and efficient way of managing applications
Server 1
VM1OS
VM2OS
VM3OS
VirtualizationServer 2
OS
Containers
APPRENDA.COM
Need more support for microservices than containers alone
Security
Container Hosting
Code Quality
Peer Discover Configuration
Changes, Supervision and Monitoring
Rolling Deployment Lib NetworkOrchestration
DockerDEVELOPMENT
PRODUCTION
APPRENDA.COM
Google Has Very Extensive Experience With Containers
•Google starts over 2,000,000,000 containers per week•15+ years of container and orchestration
R&D (Borg, Omega)• Many, many lessons learned• Distributed systems DNA:
– cgroups– mapreduce– Bigtable
"Everything at Google runs in a container… Everything"
Joe Beda, Kubernetes Founder
APPRENDA.COM
Enter Kubernetes (GA in 2015) (kubernētēs)
• Greek for “Helmsman”; also the root of the word “Governor” and “Cybernetics”
• Container orchestrator (cgroups/rkt/docker)• Supports all cloud and bare-metal
environments• Inspired and informed by Google’s experiences
and internal systems (mostly Borg)• Open source, written in Go• Declarative vs. Procedural
Manage applications, not machines!
APPRENDA.COM
PodVolume Data
Loader
Front End
Container
PodsSet of Related Containers
Tightly Coupled
Share Namespace
Ephemeral/Stateless
Share Network
Unit of Management
APPRENDA.COM
Pod Networking
DC
BA
Routable(layer 3)
No NAT(internode)
APPRENDA.COM
Labels
DC
BAapp = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2
APPRENDA.COM
Label Selectors
DC
BAapp = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2
app == portalSelector
APPRENDA.COM
Label Selectors
DC
BAapp = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2
app == portaltier == frontend
APPRENDA.COM
Label Selectors
DC
BAapp = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2
app == portalversion == v1
APPRENDA.COM
Controllers
Desired State
Current State
ActCompare
APPRENDA.COM
Replica Set (Controller)
Node 1
A
Node 2
A
Node 3
A
Node 4
A
Replicas = 4
Replicas = 4
Desired
Current
APPRENDA.COM
Replica Set (Controller)
Node 1
A
Node 2
A
Node 3
A
Node 4
A
Replicas = 4
Replicas = 3
Desired
Current
APPRENDA.COM
Replica Set (Controller)
Node 1
A
Node 2
A
Node 3
A
Node 4
A
Replicas = 4
Replicas = 4
Desired
Current
A
APPRENDA.COM
(µ)Services
app == portaltier == frontend
version == v1
Selector
A A A
Clients
SvcA
Stable Cluster-wide IP
Cluster-wide DNS name
Layer 3 Load Balancer
Updates continuously
VIP
APPRENDA.COM
(µ)Services
A A A
Clients
SvcA
A
app == portaltier == frontend
version == v1
Selector
APPRENDA.COM
• Most active GitHub project out of
3.6M
• 7K professionals list Kubernetes on
their LinkedIn profiles
• Largest number of vendors and
providers, hedges against
vendor lock-in
Kubernetes has Unprecedented Community Stats
GITHUB
36,000+COMMITS
160+RELEASES
900+CONTRIBUTORS
Top 100FORKED GITHUB
PROJECT
Top 2STARRED
GO PROJECT
Top 0.01%STARRED GITHUB
PROJECT
APPRENDA.COM
APPRENDA + GOOGLE CLOUD Moving customers to Google Container Engine
APPRENDA.COM
Who is Apprenda?
• Apprenda is cloud application platform software
• 10 years of experience in distributed systems, cloud native computing and container based application platforms
• Have been part of Kubernetes community offering solutions, including support and services, since March ‘16
APPRENDA.COM
Looked into comparable cloud native solutions:
TechnologyGoogle knows scale & it is
evident in Kubernetes
CommunityTruly open and welcoming to
all members
MarketGrowing exponentially as
orgs build better SW
APPRENDA.COM
Why Our Customers Want GKE Multicloud?
APPRENDA.COM
Customer Case StudyKubernetes on AWS and Google Container Engine Multi-cloud
• AWS Primary Cloud
• Originally Running K8S on EC2 and S3
• Migrated to GKE multi-cloud
• 60% decrease in cluster management cost
• 70% increase in developer productivity
KEY HIGHLIGHTS
• Production Kubernetes• Increased Dependence
on Mobile• Disrupted Market
APPRENDA.COM
Several Operations & Workflow Challenges
● No DevOps available to manage the cluster, web agency doing guesswork
● Teams, CI/CD spun up K8S clusters daily for development
● AWS Specific Challenges:
○ Everything under single VPC (RDS, Elasticache)
○ K8s Scripts had unintended consequences
○ EC2 Volumes filling up because of ElasticSearch setup
○ PVC leak bug with v1.2 & AWS left orphan volumes
● Sub-optimal monitoring of these temporary environments was hampering rapid
development
APPRENDA.COM
Why GKE?
● Focus on code instead of managing K8S components (etcd, master, etc.)
● Supported ancillary cloud services for K8S (storage, network, etc.)
● Closely tracks latest version● High availability architecture not left up
to user● High cpu and net performance● Easier to manage volumes,
memory/CPU utilization and find orphaned resources
APPRENDA.COM
Kubernetes is the Kernel of Container Management
Kubernetes MarketplaceInfrastructure Automation
Logging
Configuration Management
External Services
App Telemetry
Notification Services
Installation, maintenance, and
management
Storage
Middleware
Databases
Container Lifecycle Management
Policy Control
APPRENDA.COM
Key Point: Kubernetes Makes Multiple Environments Easy
Database 1 Database 2 Database 3 Database 4
App 1 (.NET) App 1 (.NET)
App 1 (.NET)
App 2 (Java)
App 2 (Java)
App 3 (Docker)
App 4 (Docker)App 3 (Docker)
App 4 (Docker)
App 4 (Docker)
PUBLIC INFRASTRUCTURE
App 3 (Docker)App 4 (Docker)
App 4 (Docker)Database 3
Database 4
DATA CENTER
App 1 (.NET)
Database 1 Database 2
App 2 (Java)
APPRENDA.COM
Key Takeaways
Lessons learned:
● Managing production Kubernetes “by hand” can be hard, GKE = “Easy Button”
● GKE has easier learning curve (compare to AWS sub-systems)
● Multi-cloud environments are easier when using “pure” Kubernetes abstractions
● Leverage expert distros/providers, support and services
APPRENDA.COM