Source Code
Working Software Build
2
x
Truck Build
Small
Monsteryouare here
3
The Monster Build Threat• Shipping
• Unit Testing
• Integration Testing
• Functional Testing
• Selenium Testing
• Synergy Harnessing
4
5
Just JIRA
6
Just JIRA(not the biggest)
7
Multi
PolandHosted
PluginsJust JIRA
8
Just JIRA
Multi
PolandHosted
Plugins
9
Wallboards
10
Build Duration• release build took 30-60 CPU hours
• supported platform matrix combinations
• 3.5 hour functional test runs
JIRA11
Long Builds Cause Low Velocity
• Context switching overhead
• Task entanglement
• Build breakage confusion
• Agile?
12
13
•optimization
•matrix culling
•maximize test power
•minimize wait time
Taming Monster Build Times
14
15
• measure first
• reduce network & disk
• RAM disks, proxies, colocation
• cut rework
• measure again
Optimization
16
•smarter admin navigation
•20% speedup
Example: JIRA Func Tests
17
•browsers
•operating systems
•databases
•libraries
Matrix Culling
18
• user based editions
• dropped ancient app servers
• dropped java 1.4
• release down to 18 hours
Example: JIRA Matrix Culling
19
Power to Wait Ratio
unit tests
functional tests
20
Be Selectively Continuous•common customer configurations
•branch vs trunk
•nightly builds
•sunday builds
21
testsfeatures
platformsbrowsersversions
ajax22
23
24
Herb Sutter http://www.gotw.ca/publications/concurrency-ddj.htm
25
26
27
Parallelism
28
Parallelism
TestingCheckout,Compile
time
agent
29
ParallelismTesting
time
agent
agentagentagentagentagentagentagentagent
agentagent
Checkout,Compile
30
The Embarrassingly Parallel Problem
vs.The Elastic Batches
of Bamboo!31
Inelegant Yet Working1532
• parametrize your test suite:
• number of batches
• current batch number
• make the suite only run subset of tests in the requested batch
Inelegant Yet Working
33
Inelegant Yet Working
34
Inelegant Yet Working
35
Need MoreComputers!
36
37
38
Reliability39
JIRA
maven 1proxy
maven 2repo
svn
nfs
release buildagent
internet
test dbjava
blogs
bamboo
license server
40
internet
test dbjava
blogs
bamboo
license server
41
$False Negatives
42
43
canary death
44
45
46
47
Maven vs. Bash
48
My technology is better
This ENDS NOW
49
• measure, optimize, measure
• use power to wait ratio
• elastic parallelization
• canary builds measure reliability
• careful tool evaluation
Monster Counter-tactics
50
Eternal Vigilance
51
Chris MountfordJIRA Dude
52