Cluster Lifecycle Landscape

Google Cloud Platform

Cluster Lifecycle LandscapeBay Area Kubernetes Meetup

August 24th, 2016Mike Danese <[email protected]>

SWEgithub: @mikedanese, twitter: @mikedanese_


Images by Connie Zhou


Kubernetes Cluster Lifecycle

What is Cluster Lifecycle?



What is Cluster Lifecycle?• Cluster is born• Cluster grows• Cluster upgrades• Cluster shrinks• Cluster dies



What is Cluster Lifecycle?• Cluster is born• Cluster grows• Cluster upgrades• Cluster shrinks• Cluster dies

Cluster Lifecycle is cross cutting.


Setting up the cluster• Choose a cloud: GCE, AWS, Azure, Rackspace, on-premises, ...• Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS,

Ubuntu, ...• Provision machines: Boot VMs, install and run kube components, ...• Configure networking: IP ranges for Pods, Services, SDN, ...• Start cluster services: DNS, logging, monitoring, ...• Manage nodes: kernel upgrades, OS updates, hardware failures...

Not the easy or fun part, but unavoidable

Deployments


The Status Quo


The Status Quo

cluster/kube-up.sh


The Status Quo

cluster/kube-up.sh is used for:• development• testing• production


kube-up has problems



Problems include but are not limited to:



Problems include but are not limited to:• totally opaque• flaky• difficult to port• polymorphic bash• jinja templates with high

cyclomatic complexity


MotivationI mean it must be possible to deploy Kubernetes since somebody did it. Why are we working on this when we could be adding more **FEATURES!!!**



MotivationBy making kubernetes easier to deploy, we lower the barrier to adoption



Use Cases


Use Cases

I want to try Kubernetes locally and get familiar with the basic concepts.


Use Cases

I want to try Kubernetes locally and get familiar with the basic concepts.

minikube!


Use Cases

I want to try Kubernetes in the cloud and kick the tires.


Use Cases

I want to try Kubernetes in the cloud and kick the tires.

kubernetes-anywhere!


Use Cases

I want to deploy Kubernetes to my existing cloud production environment andI need lot’s of configuration.


Use Cases

I want to deploy Kubernetes to my existing cloud production environment andI need lot’s of configuration.

kops!


Use Cases

I want to deploy Kubernetes to my niche and constrained production environment. There's no way anyone else is in the world is going to need the knobs that I need.


Use Cases

I want to deploy Kubernetes to my niche and constrained production environment. There's no way anyone else is in the world is going to need the knobs that I need. ?!?


Use Cases

Specifically I need:● el4, 2.6 kernel● security and compliance● airgaped● ec2-china1-gov AZ● bare-metal● f5 integration● juniper router integration● ops guy only gave me this many IPs per TOR switch


The Kelsey Challenge



From Gunshow by KC Green


We need a strategy!


Declarative and Reentrant

Deployment apis should be like Kubernetes APIs because Kubernetes APIs are great!


Kubernetes components are just applications. They needs a solution for :

• packaging/image-building

• configuration

• deployment and upgrades

• auth[nz]

• naming/discovery

• process management

• high availability

• scalability

Self Hosted


Composable and Hackable

● Deployment conflates a number of distinct processes○ Resource Provisioning (not very portable)○ Host Installation (pretty portable)○ Deploying Cluster Services (very portable)

■ kube-dns■ heapster■ kube-proxy■ fluentd/logstash■ flannel/weave/calico


Better support in Kubernetes Core


Getting the bits!

● Getting a kubelet and a compatible docker!○ apt and yum packages!

$ curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -$ cat <<EOF > /etc/apt/sources.list.d/kubernetes.listdeb http://packages.cloud.google.com/apt kubernetes-xenial-unstable mainEOF$ apt-get update$ apt-get install -y kubelet


Deploying the pod network!

● Deploying the pod network on non-cloud environments has been hard! Let’s just run these things in DaemonSets.○ Kubelet CNI runtime reconfiguration○ Flannel and Weave in DaemonSets

$ kubectl apply -f https://storage.googleapis.com/public-mikedanese-k8s/addons/weave-daemonset.yaml


Provisioning PKI

● I have to relearn all these openssl commands everytime I touch the thing. Why don’t we automate this?○ Automated certificate provisioning!○ Discovery API to distribute the public key of the cluster

root Certificate Authority!


Provisioning PKI


Provisioning PKI{ "type": "ClusterLocator", "version": "1.0", “clusterId”: “E0D87385-CE10-415F-9913-EA8388EFD80B”, "endpoints": [ "https://10.0.0.1", "https://10.0.0.2", "https://bastion.example.com/k8s/cluster1", "https://1.2.3.4", "https://1.2.3.5" ], "rootCertificates": [ "MIIDFDCCAfygAwIB….", "MIIDFDCCAfygAwIB…." ]}


<live demo>


What to look forward to. • More beta and GA!• Better Support for HA!• Better Support for Node

Upgrades!• Node Cordoning• Rescheduling and Preemption• Disruption Budget

• Better Support for Control Plane Upgrades!• etcd petset??• apiserver deployment??• kubelet checkpointing??


40

Kubernetes is Open

https://kubernetes.ioCode: github.com/kubernetes/kubernetesChat: slack.k8s.ioTwitter: @kubernetesio

open community

open design

open source

open to ideas


sig-cluster-lifecyclehttps://github.com/kubernetes/community/blob/master/sig-cluster-lifecycle

Date post:	15-Apr-2017
Category:	Technology
Upload:	mike-danese
View:	186 times
Download:	0 times

Cluster Lifecycle Landscape

Technology