Date post: | 12-Jan-2017 |
Category: |
Technology |
Upload: | docker-inc |
View: | 10,177 times |
Download: | 0 times |
Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow.Nigel HarnimanSenior Solutions Architect, CloudBees Inc
About me
Nigel Harniman
@harnimanBuild Engineer,
Devops and Architect
DevOps, Infra as Code,Continuous Delivery
“Software is eating the world”
Marc Andreessen
4
How Do You Deliver Better Software Faster?
ProdDev
5
Automation is the Key
Photo courtesy of Steve Jurvetson via Flickr
The Docker Advantage
Docker Has PotentialAn example: Software Configuration Management Space
Docker Has PotentialAn example: Software Configuration Management Space
The New World Order: Containers Codify OS Config
9
ProdDev QA Staging
DEV Server/VM QA Server/VM STG Server/VMPROD Server/VM
<PROD OS config><STG OS config><QA OS config><DEV OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
App<code>
<APP OS config>
So is this how I build a Docker Image?
10
Jenkins & Docker
How Can You Use Jenkins & Docker Together?
+
How Can You Use Jenkins & Docker Together?
1. Run Jenkins Masters & Slaves in Docker
2. Build, Test, & Deploy Docker Images from Jenkins
1. Run Jenkins Masters & Slaves in DockerDocker (Cloud) – use Docker images as standardized build environments to improve isolation and elasticity
Docker Custom Build Environment – specify customized build environments as Docker containers
CloudBees Docker Shared Config – manage Docker (or Swarm) host configuration centrally in CloudBees Jenkins Operations Center
2. Build, Test, & Deploy Docker Images from JenkinsBuild and Publish – build projects that have a Dockerfile and push the resultant tagged image to Docker Hub
Docker Traceability – identify which build pushed a particular container and displays the build / image details in Jenkins
Docker Hub Notification – trigger downstream jobs when a tagged container is pushed to Docker Hub
Jenkins Workflow & Docker
Jenkins Workflow PrimerJenkins powered CD pipelines
Jenkins Workflow
ProdDevPerf Test
BuildCommit Selenium Test Stage Deploy
Sonar Test
Pipelines Need: Branching Looping Restarts
Checkpoints Manual Input
??
Key Workflow Features
18
Entire flow is one concise Groovy script using Workflow DSL• For loops, try-finally, fork-join …
Can restart Jenkins while flow is running
Allocate slave nodes and workspaces• As many as you want, when you want
Stages throttle concurrency of builds
Human input/approval integrated into flow
Standard project concepts: SCM, artifacts, plugins
Jenkins Workflow + Docker
20
Pipeline Stages
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
21
Build, unit test and package
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
Build, unit test and package
stage 'Build App’
node('docker') {
docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {
mkdir –p /data/mvn
writeFile file: 'settings.xml', text: ”(………)"
git 'https://github.com/cloudbees/mobile-deposit-api.git’
sh 'mvn –s settings.xml clean package’
…
Specify the Stage Name
Specify the slave labelCustom Build Env Mount volume from
slave
.m2 repo locationco and build
Defining a Docker SlaveSpecify Image as
templateAssign labels
24
Test the app
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
Test the app
node('docker') {
docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {
…
stage 'Sonar analysis’
sh 'mvn -s settings.xml sonar:sonar’
stage 'Integration-test’
sh 'mvn -s settings.xml verify’
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
…
In same env as build
Sonar tests
Run API Tests
26
Build, test and publish Docker image
Build and Unit Test App
Test Docker Image
Publish Docker Image
SCM Checkoutmvn package
mvn sonar:sonarmvn verify
docker build
docker Tag
docker run
notifycucumber
war
img
Sonar Analysi
s
Prepare Release
Build Docker Image
Int Test
docker push
image.inside withServer
Build, test and publish Docker image
docker.withServer('tcp://192.168.99.100:2376', 'slave-docker-us-east-1-tls'){stage 'Build Docker image’
def mobileDepositApiImage
dir('.docker') {sh "mv ../target/*-SNAPSHOT.jar mobile-deposit-api.jar”
mobileDepositApiImage = docker.build "harniman/mobile-deposit-api:$
{buildVersion}”
} stage 'Test Docker image’
container=mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080”)
sh "curl
http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \......// insert cucumber tests here
stage 'Publish Docker image’
withDockerRegistry(registry: [credentialsId: 'dockerhub-harniman']) {mobileDepositApiImage.push()
}
}
Bind to docker host
Change directory
Launch container
Build docker image
Bind to registryPush image
Submit traceability report
28
Tagged Version
aTagged Image in Docker Huba
Traceability
30
Traceability
Builds on existing Jenkins artifact traceabilityAllows the tracking of the creation and use of Docker containers in Jenkins and their future use.Combine with artifact fingerprinting for a comprehensive solutionEach Build shows the image fingerprints created
Identify which build pushed a particular container and display the build / image details in Jenkins
Image fingerprints
31
Traceability – registering eventsJenkins can track actions against this image such as:
• Creating a container• Container events such as start/stop
To achieve this, it is necessary to call the Traceability API – see $(JENKINS_URL)/docker-traceability/api/There are two endpoints to submit events to:/docker-traceability/submitContainerStatus
Allows to submit the current container status snapshot with a minimal set of parameters. Outputs of docker inspect $(containerId) can be directly submitted to Jenkins server using this command.
/docker-traceability/submitReport
Submits a report using the extended JSON API. This endpoint can be used by scripts to submit the full available info about the container and its environment in a single command.
32
Traceability – registering events - exampleWorkflow usage example:
container = mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080")
sh "curl http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \
--data-urlencode status=deployed \--data-urlencode inspectData=\"\$(docker inspect $container.id)\" \--data-urlencode environment=test \--data-urlencode hostName=mymac \--data-urlencode imageName=harniman/mobile-deposit-api"
Spin up container
Notify Jenkins
33
Docker Traceability View
Docker TraceabilityContainer
Container Use View
34
Deployment Events
Link to Build
Dockerhub Notifications
36
Docker Hub NotificationTrigger downstream jobs when a tagged container is pushed to Docker Hub
The Docker Hub Notification Trigger plugin lets you configure Jenkins to trigger builds when an image is pushed to Docker Hub. E.g. to run verification for the container.
What are the steps
Set up a WebHook Account for Notification
Set up your Docker Registry to make callbacks on Image events
Set up your builds
37
Docker Hub Notification – Docker Registry WebhookIn the format:http://<user>:<token>@<jenkins_url>/dockerhub-webhook/notify
38
Docker Hub Notification – Job Set up
Configure Trigger
In Conclusion
40
Docker and Jenkins with Workflow is the proven CD Platform
+
TESTING
STAGING
PRODUCTIONWorkflow CD Pipeline Triggers:• New application code (i.e. feature, bug, etc.)• Updated certified stack (security fix in Linux, etc.)
… will lead to a new gold image being built and available for…… TESTING
… STAGING… PRODUCTION
All taking place in a standardized/similar/consistent environment
<OS config>
Company“Gold”
Docker Img(~per app)
App<code>
(git, etc.)<OS config>
Certified DockerImages
(Ubuntu, etc.)
Jenkins Workflow
CloudBees: Leading the Way for Docker and CD
Docker Workflow – Provides first-class support for Jenkins Workflow to build real world CD pipelines for containerized applications using Jenkins and Docker
Build and Publish – Builds projects that have a Dockerfile and pushes the resultant tagged image to Docker Hub
Docker Hub Notification – Triggers downstream jobs when a tagged container is pushed to Docker Hub
Docker Traceability – Identifies which build pushed a particular container that is running in production and displays that on the Jenkins builds page
Docker – Uses Docker containers as standardized build environments to improve isolation and elasticity – Dockerized Build Slaves
Docker Custom Build Environment – Specifies customized build environments as Docker containers
Getting started
Docker plugin documentation:http://documentation.cloudbees.com/docs/cje-user-guide/docker-workflow.html
Workflow tutorial:https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md
Example Source Codehttps://github.com/harniman/mobile-deposit-api/blob/master/flow.groovy
How Do You Manage CD at Enterprise Scale?
43
CloudBees Jenkins PlatformJenkins at Enterprise Scale for CI and CD
Thank you!Nigel Harniman@[email protected]