59
Title TextTitle Text
The State of Kubernetes Development ToolingOctober 2019
Kubernetes the hardKubernetes the hardwayway falsely implies that falsely implies thatthere's an easy way.there's an easy way.
@pczarkowski
@ellenkorbes
@ellenkorbes
■ Run on Kubernetes.■ Are not Kubernetes itself. E.g. CRDs & operators.
Who is this talk for?@ellenkorbes
Developers working on apps that...
■ Their pre-commit workflow.■ The part of their day where code gets written.
Who is this talk for?
■ Their CI/CD pipelines.■ Ops stuff not directly related to writing code.
But not necessarily...
Developers looking to improve...
@ellenkorbes
You put the whole appYou put the whole appin one container?!in one container?!
@7hunderbird
@ellenkorbes
■ Provisioning■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug■ Git & CI■ Ship to prod. & CD■ Observability
DevelopmentCycle:
@ellenkorbes
Pre-CommitStage:
■ Provisioning■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug■ Git & CI■ Ship to prod. & CD■ Observability
@ellenkorbes
■ Configuration files
■ Bootstrap
■ Write code
■ Build
■ Deploy to dev
■ Test
■ Debug
Setup
Dev
Feedback
@ellenkorbes
■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug
Wishlist■ Manageable size■ Create them for me?
@ellenkorbes
■ Setup tasks (like migrations)
@ellenkorbes
■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug
Wishlist
■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug
■ Automate docker/kubectl cycle■ Short feedback loop■ Unobtrusive■ Hot reload■ Easy to on-board■ Dependency-aware
@ellenkorbes
Wishlist
■ Do testing■ Highlight failures■ End-to-end tests
@ellenkorbes
■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug
Wishlist
■ Log streaming■ Highlight issues■ Allow breakpoints■ Not tracing
@ellenkorbes
■ Configuration files■ Bootstrap■ Write code■ Build■ Deploy to dev■ Test■ Debug
Wishlist
Maybe containers wereMaybe containers werea mistake.a mistake.
@jessfraz
@ellenkorbes
@ellenkorbes
■ Debug■ Connect■ Build & Deploy
Cycle
Categories
■ Draft■ Garden■ Skaffold■ Tilt■ Jenkins X■ Forge■ Helm
@ellenkorbes
Tools■ ksync■ kubefwd■ Okteto■ Telepresence■ inlets■ Squash■ Stern
■ Debug■ Connect■ Build & Deploy
Cycle
Categories
■ kubeval■ yq■ kubectx■ kubens■ ytt■ kpxg■ kbld■ kapp■ kwt■ ...
Others...
@ellenkorbes
@ellenkorbes
Trivia:Which of theseisn't a real tool?
■ kubeval■ yq■ kubectx■ kubens■ ytt■ kpxg■ kbld■ kapp■ kwt■ ...
Others...
Kubernetes:Kubernetes:complicating simplecomplicating simplethings since 2014.things since 2014.
@texmandie
@ellenkorbes
Debug■ kubectl■ Squash■ Stern
■ Easily shows you the status of your pods,deployments, services.
■ Learn it.■ kubectl logs --follow -n [namespace] [pod]
Notes@ellenkorbes
@ellenkorbes
Debug■ kubectl■ Squash■ Stern
■ Steps and breakpoints on deployedcontainers!
■ No more Printf debugging �■ IDE or CLI■ Delve, gdb, Java, Node, Python■ Fun!
@ellenkorbes
Notes:Debug■ kubectl■ Squash■ Stern
@ellenkorbes
Debug■ kubectl■ Squash■ Stern
■ Streams logs from multiple pods■ Color-coded■ Easy to filter■ It's the best■ I love it �■ Use it!
@ellenkorbes
Notes:Debug■ kubectl■ Squash■ Stern
@ellenkorbes
Debug■ kubectl■ Squash■ Stern
■ "[...] think of it as a very fancy shell scriptthat does a docker build, docker push,kubectl apply [...]" — rdldw
■ Builds and deploys! Doesn't watch.■ Requires YAML, Dockerfile, Forge config.■ Allows profiles e.g. dev/canary/production.■ Development seems stopped �
@ellenkorbes
Notes:Connect■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
■ Package manager for Kubernetesapplication structure.
■ Install and uninstall off-the-shelf apps. Thinkapt-get, brew, npm, pip, gem.
■ App lifecycle management. Deploy a service'snew version, roll back, delete.
■ Omnipresent: Haven't used it? You will.■ Pro & con: Go templating on configs.
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
■ Reverse proxy & websocket tunnel to exposeinternal endpoints to the internet.
■ Similar to ngrok or Argo Tunnel, but free andopen source.
■ Has a Kubernetes operator so you canseamlessly expose a local Kubernetes cluster.
■ Use it to share works in progress, set upwebhooks in development, and so on.
■ ACHTUNG! You're exposing internals to theinternet. Be mindful of security.
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
@ellenkorbes
■ inlets
■ Watches your file system■ Syncs local files with files in your cluster■ Straight forward and easy to use■ Built on top of Syncthing■ You do need a watcher inside the
container to act on the updated files!
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
■ Forward all possible ports—asopposed to kubectl which forwardsindividual ports.
■ Access any service locally like a podin the cluster
■ Easy! Fun! Buy it now! (Just kidding.)
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
■ Swaps a deployment for adevelopment environment.
■ Syncs & watches files, gives you aremote terminal, does port forwarding.
■ Ideal to build admission controllers,operators, things deeply integratedwith k8s.
■ One service at a time.
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
■ Hot swaps a running pod and creates alocal shell where the network is proxied intothe cluster.
■ Run app locally; it acts as if in the cluster.■ Use local IDE, debugger, etc.■ Vs. Squash: Single instance. Might cause
conflicts like env vars and docker version.■ Vs. Okteto: Local machine pretends tobe a node vs. moving dev into the cluster.
@ellenkorbes
Notes:■ Forge■ Helm■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
Connect
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
■ Generates config files: Great if you're newto containers or Kubernetes! Helm based.
■ Supports bootstrapping tasks.■ Works per-service: You must "draft up"
for every service.■ "The team originally working on Draft
have to put it on the backburner whilefocusing on efforts like Brigade, Helm 3,and CNAB."
■ No hot reload!
Notes:
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
■ No Kubernetes manifests required.(Unless you want to!)
■ Allows in-cluster builds. LocalKubernetes, Docker, minikube notrequired.
■ Project awareness: It tracks build anddeploy dependencies.
■ Tests are first-class citizens. Locally & CI.■ Pluggable: Custom module types for e.g.
OpenFaaS, Java, Terraform. (Soon alsoGo!)
Notes:Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
■ Something about provisioning.■ Something about CI.■ Uses Skaffold and ksync in the
background.■ Has build packs that generate config
files similarly to Draft! �
Notes:Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
■ Automatically builds & deploys!■ For those already familiar with k8s, it
doesn't require learning new concepts.■ Requires k8s manifests! Not for
beginners.■ Easy set up if you already have them.■ No support for dependencies, tests, or
bootstrapping.
Notes:Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
■ Like Skaffold: Requires k8s manifests;easy set up if you already have them.
■ Gorgeous UI, dashboard, and logging.■ Vs. Skaffold? Skaffold's concepts map
almost directly into Tilt. ■ Starlark configs!
Notes:Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
@ellenkorbes
Build &Deploy■ Draft■ Garden■ Jenkins X■ Skaffold■ Tilt
Toolchains:
■ Helm■ kubectl
■ Draft ☠■ Garden■ Jenkins X■ Skaffold■ Tilt
■ Forge ☠■ inlets■ ksync■ kubefwd■ Okteto■ Telepresence
■ kubefwd■ Squash■ Stern
Debug:Connect: Build&DeployCycle:
@ellenkorbes
Omnipresent:
It works on my cluster...It works on my cluster...@maeddes
@ellenkorbes
Still missing:■ Better docs!■ Consistent naming for
things across different tools.■ Better support for tests!■ ■ Beginner friendliness.■ Clarity. A cohesive story. ?
@ellenkorbes
(ls *.go | entr r ./mytest.sh)
■ Draft → ■ entr → ■ Forge → ■ Garden → ■ Helm → ■ inlets → & ■ Jenkins X → ■ ksync → ■ kubectl 'Debug Services' →
■ kubefwd → ■ Okteto → ■ Skaffold → ■ Squash → ■ Stern → ■ Telepresence → ■ Tilt →
draft.sheradman.com/entrproject/forge.shgarden.io
helm.shgithub.com/inlets/inlets inlets-operator
jenkins-x.iogithub.com/vapor-ware/ksync
kubernetes.io/docs/tasks/debug-application-cluster/debug-service/
kubefwd.comokteto.comskaffold.devgithub.com/solo-io/squash
github.com/wercker/sternwww.telepresence.io
tilt.dev
Links:@ellenkorbes
@krisnova
You can't have aYou can't have aclusterf!$& without aclusterf!$& without a
cluster.cluster.
@ellenkorbes
[email protected] | @ellenkorbes
