Post on 25-Jan-2017
transcript
(A Practical Guide to)Continuous Delivery with Containers
Daniel Bryant @danielbryantuk
01/05/2023 @danielbryantuk
Setting the scene…• Continuous delivery is a large topic
• Focusing on the process and tooling• Rather than each explicit step
• My O’Reilly mini-book will provide step-by-step instructions
• Assuming basic knowledge of Docker
01/05/2023 @danielbryantuk
Today…• Continuous Delivery (CD)
• The impact of containers on CD
• Creating a container pipeline
• Migrations: Architectural guidance
• Lessons learned the hard way
01/05/2023 @danielbryantuk
@danielbryantuk• Chief Scientist at OpenCredo, CTO at SpectoLabs
• Agile, architecture, CI/CD, DevOps
• Java, Go, JS, microservices, cloud, containers
• Leading change through the application of technology and teams
• London Java Community Associate• InfoQ Editor, DZone MVB, O’Reilly…• Conference regular: Devoxx, JavaOne, QCon…
01/05/2023 @danielbryantuk
Continuous Delivery
01/05/2023 @danielbryantuk
Continuous Delivery
• Produce valuable and robust software in short cycles
• Optimising for feedback and learning
• Not (necessarily) Continuous Deployment
01/05/2023 @danielbryantuk
Creation of a build pipeline is mandatory for continuous delivery
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
The Impact of Containers on CD
01/05/2023 @danielbryantuk
Containers: Expectations versus reality
“DevOps”
01/05/2023 @danielbryantuk
Container technology• OS-level virtualisation• cgroups, namespaces, rootfs
• Technology to package and execute software
• The container image becomes the source of truth
• Mechanical sympathy is vital
01/05/2023 @danielbryantuk
We’ll focus on Docker today• Docker images are built via a Dockerfile
• docker build –t danielbryantuk/test:1.4
• Publish images• docker push danielbryantuk/test:1.4
• Download images• docker pull danielbryantuk/test:1.4
• Run an image as a container• docker run –p 80:80 danielbryantuk/test:1.4
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Quick interuption: Microservices…• Containers and microservices are
complementary
• Not covering details for deploying microservices today
• But if you are interested:• Consumer-based contracts • Service virtualisation• Synthetic transactions and semantic
monitoring https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Creating a Pipeline for Containers
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Make your dev environment like production• Develop locally or copy/code in container
• Ensure language runtime/SDK is synced
• Must build/test containers locally• Perform (at least) happy path tests before
pushing code• All tests should be runnable locally
01/05/2023 @danielbryantuk
What to put in the Dockerfile
• OS choice• Exposed to OS (often implictly?)• Choose lightweight OS if possible e.g. Alpine, Debian
Jessie
• Configuration
• Build artifacts
• Exposing ports
• Java• JDK vs JRE• Oracle vs OpenJDK
• Golang• Statically compiled binary
• Python• Virtualenv
01/05/2023 @danielbryantuk
Please talk to the sysadmin people:Their operational knowledge is invaluable
01/05/2023 @danielbryantuk
Different dev and test containers?• Test container• Full OS (e.g. Ubuntu)• JDK • Test tools• Test data
• Easy to see configuration drift
• Interesting ONTEST proposal by Alexi Ledenev
http://blog.terranillius.com/post/docker_testing/
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Building images with Jenkins• Standard Jenkins Java
• Gradle or Maven• SonarQube for code quality
• (Optionally) push to artifact repo• Nexus and Artifactory support Java artifacts
and Docker images
• Build Docker Image• Cloudbees Docker Build and Publish Plugin
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Storing in an image registry (DockerHub)
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
A little context…
01/05/2023 @danielbryantuk
Introducing Docker Compose
01/05/2023 @danielbryantuk
Testing: Jenkins Pipeline as Code
01/05/2023 @danielbryantuk
Testing: Jenkins Pipeline as Code
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Jenkins ‘BlueOcean’ Beta
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Docker Compose & Jenkins Pipelines
01/05/2023 @danielbryantuk
Testing: Functional• Automate all the things!• Deploy to realistic environments
• API-driven functional• REST-assured
• UI-driven functional• Selenium• Serenity BDD• Geb
01/05/2023 @danielbryantuk
Testing: NFRs• Execution (runtime)• Security• Observability
• Evolvability (static)• Testabillity• Maintainability• Scalability• Extensibility
01/05/2023 @danielbryantuk
Testing: NFRs• Security testing • Findsecbugs• OWASP Dependency check• Bdd-security (OWASP ZAP) / Arachni • Gauntlt / Serverspec• Docker Bench for Security / AQUA
• Performance and Load testing • Gatling / Jmeter• Flood.io
01/05/2023 @danielbryantuk
Special mention: Container security testing
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Special mention: Fault tolerance testing
01/05/2023 @danielbryantuk
Fault tolerance
techblog.netflix.com/2016/10/netflix-chaos-monkey-upgraded.html github.com/tomakehurst/saboteur
01/05/2023 @danielbryantuk
Hoverfly• Lightweight Service virtualisation • Open source (Apache 2.0)• Go-based / single binary • Written by @Spectolabs
• Flexible API simulation• HTTP / HTTPS• Highly performant
01/05/2023 @danielbryantuk
• Middleware• Remove PII• Rate limit• Add headers
• Middleware• Fault injection• Chaos monkey
01/05/2023 @danielbryantuk
01/05/2023 @danielbryantuk
Deploy• Test environments should represent
production (as much as possible)
• Fan-in infrastructure pipelines with applications as soon as possible
• Ask yourself: Do you really want to create a container platform?
01/05/2023 @danielbryantuk | @spoole167 49
Don’t underestimate the value of PaaS…
01/05/2023 @danielbryantuk
Post-deployment
01/05/2023 @danielbryantuk | @oakinger
When bad things happen, people are at the center
01/05/2023 @danielbryantuk
Monitoring is vital with continuous delivery• Host monitoring
• Container monitoring
• Application monitoring
https://github.com /Kentik/docker-monitor
01/05/2023 @danielbryantuk
Migrations: Architectural Guidance
01/05/2023 @danielbryantuk
Containerise the monolith?• For
• We know the monolith well
• Allows homogenization of the pipeline and deployment platform
• Can be a demonstrable win for tech and the business
• Against
• Can be difficult (100+ line scripts)
• Often not designed for operation within containers, nor cloud native
• Putting lipstick on a pig?
01/05/2023 @danielbryantuk
Key lessons learned• Conduct an architectural review
• Architecture for Developers, by Simon Brown• Architecture Interview, by Susan Fowler
• Look for data ingress/egress• File system access
• Support resource constraints/transience• Optimise for quick startup and shutdown • Evaluate approach to concurrency• Store configuration (secrets) remotely
01/05/2023 @danielbryantuk
Containers and cloud: Design for failure• Distributed Computing Principles• Jeff Hodges ‘Distributed Systems’ (bit.ly/1FeaVtt) • Scalable Web Architecture (bit.ly/1tt703O)• ‘For young bloods’ (bit.ly/1pKVepz)
• Design patterns• Timeouts / retries• Bulkheads / circuit-breakers
01/05/2023 @danielbryantuk
Using containers does not obviate the need for good architecture
01/05/2023 @danielbryantuk
https://speakerdeck.com/caseywest/containercon-north-america-cloud-anti-patterns
01/05/2023 @danielbryantuk
Lessons Learned the Hard Way
01/05/2023 @danielbryantuk
Miscellaneous (but vital)• Beware of the ‘latest’ Docker tag• Properly version your containers
• Metadata is vital• Labels can be valuable• h/t MicroBadger
• www.notonthehighstreet.com case study and learnings• http://bit.ly/1PMlpIL
01/05/2023 @danielbryantuk | @spoole167 62
Mechanical sympathy: Docker and Java• Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead• Account for native thread requirements e.g. thread stack size (Xss)• Default fork/join thread pool sizes (based from host CPU count)• Watch out for ulimits
• Entropy • Host entropy can soon be exhausted by crypto operations• –Djava.security.egd=file:/dev/urandom• Be aware of security ramifications
01/05/2023 @danielbryantuk
Summary
01/05/2023 @danielbryantuk
In summary• Continuous delivery is vitally important in modern architectures/ops
• Container images must be the (single) source of truth within pipeline
• Mechanical sympathy is important (assert properties in the pipeline)• We’re now bundling more into our artifact (e.g. an OS)• Not all developers are operationally aware
• The tooling is now becoming stable/mature• We need to re-apply old CD practices with new technologies/tooling
01/05/2023 @danielbryantuk
Bedtime reading
01/05/2023 @danielbryantuk
Thanks for listening
• Any questions?
• Feel free to contact me• @danielbryantuk• daniel.bryant@opencredo.com