49
Hudson Project
Open-source CI server Written in Java but used for all kinds of
languages
Emphasis on ease of installation and use “java -jar hudson.war” execution Or your OS-specific package Configure everything from browsers
2
https://hudson-labs.org/
3
Ecosystem
The real reason why Hudson thrives The same reason why Eclipse thrives Extensibility
330+ community-developed public plugins Up from 230+ last year in JavaOne
Growing Hudson Adoption
Eclipse Community Survey 9.1% in 2009 -> 21.8% in 2010
Polling by John Ferguson
4
2008 2009 2010
Hudson 39% 46% 67%
CruiseControl 18% 12% 3%
Bamboo 7% 5% 8%
TeamCity 9% 14% 7%
Install Base is Growing
2008
-12
2009
-02
2009
-04
2009
-06
2009
-08
2009
-10
2009
-12
2010
-02
2010
-04
2010
-06
0
5000
10000
15000
20000
25000
6
Growing Hudson Adoption
Source: indeed.com
7
InfraDNA
I left Oracle this April and started a company around Hudson
Support Subscriptions On open-source Hudson + plugins Or on ICHCI: a commercial value-added
distribution Consulting
Troubleshooting, custom plugins, etc.
Visit us at partner kiosk outside exhibit floor
http://infradna.com/
First steps to CI
Hudson notices that there is a change
Check out the source code Do some beefy computation
Builds Run tests
Record and publish results Report to people
Only when necessary8
9
But You Can Do So Much More!
Once you automate, you can do a lot
Yet average Hudson users only use very small part of Hudson
Hence this talk
10
Interconnecting Jobs aka Workflows
11
Inter-Project Interactions
You can host many projects on 1 Hudson Lots of ways to have them interact with
each other
Increased parallelism is good
12
Building Block #1: triggering When a build is done, kick off something
else
“upstream”: the build that’s kicking off “downstream”: the build being kicked off
Triggering is asynchronous A large graph forms a cascading behaviour Upstream doesn’t wait for downstream Downstream can tell who is the upstream
Foo #10 Bar #153
Test/Build Separation
Decompose test run into suite runs Trigger them all from build
Build #10
Test X
Test Y
Test Z
Build #11 Build #12 …
time
13
Test/Build Separation Technique
Do not recompile the bits during tests Instead, copy the pre-built bits
Use the copy artifact plugin Or wget/curl via permalink
14
15
Build/Deploy Separation
1 job to build software, another to deploy it
Deployment can be rerun without rebuild Failed build, new server, etc.
Helps promote separate responsibilities
16
Building Block #2: join plugin
Fan out, and then join back
Build #10
Test X
Test Y
Report Generation
Build #11 Build #12 …
time
17
Join Configuration
18
Building Block #3: Fingerprints
MD5 checksums recorded during builds x.jar in product Z #112 had a bug.
Where did it come from? Did it pass all tests?
Checksum Origin Filename
12ab34cd56ef78
componentX-nightly #16
x.jar
12ab34cd56ef78
component-X-test #39 x-1.0-SNAPSHOT.jar
0bee89b07a24c componentX-nightly #17
x.jar
0bee89b07a24c componentX-test #40 x-1.0-SNAPSHOT.jar
12ab34cd56ef78
productZ-integration #112
x.jar
19
Fingerprints
The same mechanism works for … Tracking 3rd party dependencies Tracking the version of the build tools
used
Fingerprinting is critical when you have inter-related jobs Used to show relationship in UI Used to aggregate test reports
20
Build Promotion
Sometimes plain-vanilla triggering is not desirable
Example: Build X Run smoke test suites X1 and X2 If they both pass, deploy X to QA site Run QA test If it pass, deploy X to staging site
21
Build Promotion
Hudson’s solution: build promotion Model
Define a promotion level▪ “Cleared Smoke tests”
Define a promotion criteria▪ “Successful completion of X1test and X2test”
Define what happens after promotion▪ “Tag the source tree and trigger Xqa”
22
More Gimmicks in Build Promotion
Define multiple promotions E.g., “cleared smoke test” & “cleared QA
test” Convenient for exposing builds to
progressively more expensive work
Promotion criteria can be “manual approval” We often can’t automate away
everything!
Last promoted build gets a permalink
Demo
24
Getting More From Your Maven Builds
Photo: http://www.flickr.com/photos/seeminglee/4556156477/
25
Maven Integration Highlights
POM comprehension Simplified configuration
More gets inferred automatically Snapshot dependencies among
builds
26
Dealing With Large Maven Build
Parallel module build Most projects have short but wide
dependency chain
Let Hudson build on module-by-module basis
Wherever possible, build in parallel
27
Dealing With Large Maven Build
Incremental builds Build changed modules and their
descendants SCM determines what has changed
28
Private Maven Repository Cause
~/.m2/repository is not concurrent safe Multiple builds of the same project▪ Different branches, test environments, JDK, etc.
Problem Artifacts overwrite each other Artifacts change under the carpet
Solution Allocate ~/.m2/repository per job Consider a repository manager
29
After-the-fact Deployment
Default “mvn deploy” behavior harmful Think about a failure in the middle
Better to let the build complete, then deploy
Module1
Module2
Module3
Module4
Module5
30
Really After-the-fact Deployment
Deploy much after the build has finished
31
Matrix Project
Photo: http://www.flickr.com/photos/amarilloposters/4380080793/
32
Motivation
You often do the same thing with slight variations Compile C++ code for different
platforms Test with different browsers Or more generally, think of it asfor (x in [a,b,c]) {
for (y in [d,e,f]) { … doSomethingWith(x,y,z,..); … }}
33
Model
Define axes One axis ≈ one for loop
Choose from pre-defined types of Axis Slave axis: pick slaves by their names or
their labels▪ e.g., linux, solaris, and windows
Generic axis: arbitrary values exposed as environment variables
34
Matrix Project Gimmicks
Filtering Otherwise combinations increase
exponentially Not all combinations make sense Use boolean expression to trim down the
size
Or tell Hudson to cut the workload to N%▪ Hudson will thin out the combinations by
itself
(label==“windows”).implies(browser==“iexplore”) &&(label==“mac”).implies(browser==“safari”)
Demo
36
Doing Selenium Tests on Hudson
37
Selenium Grid
Automatically installs Selenium grid binaries On all Hudson slaves
Automatically start Selenium RCs and connect them
Starts Hub on Hudson master port 4444
Hudson cluster now accessible as Selenium RC farm from anywhere
Selenium Grid
Use Hudson slaves asSelenium RC nodes
38
Hudson master
(selenium hub)
Hudson slaves
39
Choosing Environment
Use Hudson’s labels to choose the slave that runs the browser *firefox
Run Firefox somewhere (but no guarantee that the chosen slave has it)
windows:*firefoxPick a slave that has the “windows” label, then run Firefox
windows&sfbay:*firefoxPick a slave that has both “windows” and “sfbay” label, then run Firefox
Testing with multiple browsers Use “matrix project” type Run the same tests with different browsers Aggregate the results
40
Firefox tests
IE tests
Safari tests
41
Selenium Grid w/Hudson Benefits
No sysadmin overhead for setting up Selenium RCs Especially nice on heterogeneous
network
Reuse your existing investment in Hudson cluster
42
Launching browsers inside Hudson
Hudson typically runs without GUI aka “Headless”
Browsers need GUI
So you need to make GUI environment accessible to Hudson and its slaves
43
On Unix
Use Hudson xvnc plugin Implements X protocol w/o a real GUI Needs xvnc binaries to be installed
everywhereapt-get install vnc4server
Hudson will start/stop xvnc and control $DISPLAY automatically
44
On Windows
Windows has the concept called “WindowStation” Equivalent of X server in Windows WinSta0 is the only interactive desktop
So even headless processes can launch browsers. In theory.
45
On Windows, in practice
GUI programs can hang with a dialog For example, IE8 setup wizard
Thus it’s often easier to just let them access WinSta0
46
On Windows, really in practice
But only SYSTEM user can interact with desktop Different user account creates different
environments that often trip other things Fallback
Configure Windows for auto-login Use start up script to start JNLP slaves
47
On Mac
User that runs Hudson slaves needs to be also logged in to the desktop There appears to be no way to run GUI
processes headlessly
Conclusions
If you aren’t using Hudson yet… What are you waiting for?
If you are using Hudson… You probably can get more out of it
48
