Post on 24-Dec-2015
transcript
Waterloo Agile Lean P2P Group
Continuous IntegrationIf it is so great why can't we make it work here?
Leon KehlFebruary 16, 2010Waterloo Ontario
Agenda
What is Continuous Integration and its History? Demonstration of Continuous Integration Technology Continuous Integration & Lean Production People and the Laws of Physics Continuous Integration Case Study
Martin Fowler's Definition
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
History
May date to the 60's when the IBM 360 team rebuilt the code base four times a day.
Continuous integration was one of the practises of the C3 team which included Kent Beck in 1996 that introduced Extreme Programming.
Martin Fowler wrote an article in 2000 which included the following: “The tests are integrated into a continuous integration and build process which yields a highly stable platform for future development.”
CruiseControl 1.0 was released in March 2001.
Recommended practices
Maintain a code repository
Automate the build Make the build self-
testing Everyone commits
every day Every commit (to
mainline) should be built
Keep the build fast Test in a clone of the
production environment
Make it easy to get the latest deliverables
Everyone can see the results of the latest build
Automate deployment
Today
Wikipedia lists 35+ applications. Originally written in Java, Cruisecontrol has
been ported to .NET and Ruby and there is now a commercial version.
Most are open source like Cruise Control but a number of commercial versions exist.
CruiseControl driven primarily by a config.xml file, others more graphical
Quick Demonstration of CruiseControl
Downloaded from http://cruisecontrol.sourceforge.net/download.html
Bundled with versions of Jetty and Ant
Just unzip and run
Controlled through config.xml file
Granddaddy of tools, but easy to install and maintain. Newer applications easier to configure with more bells and whistles
Features
Monitors source repository for changes Builds based on changes or time based Automated build and test Emails success/failure Maintains history of builds Provides web interface to system
Advantages Easy to revert if bug
encountered Detecting integration
problems Early warning for broken
code Immediate unit testing Working version constantly
available Focus developers on
developing functional, quality code
Disadvantages Initial setup time required Well-developed test-suite
required for best utility
So what's the problem?
What are the goals of a typical software build process?
What are the goals of a typical software build process?
Deliver a controlled build of the product to the user or testers
Identify changes from previous versions for updates
Traceability back to source code versions May include regular nightly builds to ensure
product can be delivered
What are the goals of a typical software build process?
Deliver a controlled build of the product to the user or testers
Identify changes from previous versions for updates
Traceability back to source code versions May include regular nightly builds to ensure
product can be delivered
Focus is typically on delivering final build to customer, not on supporting the development of product or on interim quality standards
Is Continuous Integration Equivalent to Kanban?
What lies beneath the water?
Software - Laws of Physics
Inertia is the resistance of any physical object to a change in its state of motion.
For software systems; as a system is modified, its disorder, or entropy, always increases. This is known as software entropy
Example application of CI
Existing legacy check processing application ported to Windows “combined” with another legacy application
“Unit tests” meant updating a 100 page Word document with a manual test for feature
Testing changes involved going to test lab and running checks with physical hardware, setup could take an hour or longer
Severe case of Software Entropy Significant problems in field, fixing issues and adding
support for high speed transports Similar situation to C3 project
Greenfield redevelopment
Able to utilize Agile methodologies, part of selling redevelopment to management
Incorporated continuous integration from day 1 Build system, unit testing etc. were never
separate deliverables but part of the process. Continuous integration included automated
build, unit tests, integration tests and ran in under 5 minutes
Nightly builds included documentation, code coverage and were added midproject
Successful application of CI
Metaphor for project was eating an elephant in this case 110,000 lines of legacy code
Version delivered for acceptance testing after 8 months
Fairly long acceptance testing and rollout Approaches and techniques were applied in
limited situations elsewhere
Continued support for Agile
Senior leadership supportive of Agile, created a Build committee to improve build processes
Several other teams were also working on continuous integration using variety of tools
Support for application transitioned from contract resources to all full-time for this project
Agile consulting firm was engaged to analyze current processes
Manifesto for Agile Software Development
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Manifesto for Agile Software Development
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Software - Laws of Physics Revisited
Inertia is the resistance of any physical object to a change in its state of motion.
For software systems; as a system is modified, its disorder, or entropy, always increases. This is known as software entropy
Iceberg Below the Surface
Behaviour
Mindsets
Assumptions
Habits
Unwritten rules
Culture
Trivial Pursuit
Who knows what happened to the original C3 project at Chrysler?
Trivial Pursuit
The plan was to roll out the system to different payroll 'populations' in stages, but C3 never managed to make another release despite two more years' development. The C3 system only ever paid 10,000 people (target was 85,000)
Chrysler was bought out by Daimler-Benz in 1998. DaimlerChrysler stopped the C3 project on 1 February 2000.
Frank Gerhardt, a manager at the company, announced to the XP conference in 2000 that DaimlerChrysler had de facto banned XP after shutting down C3; however, some time later DaimlerChrysler resumed the use of XP.
Law of Inertia
Build committee struggled to get traction, focused on hardware, software tools
Senior management to kickstart process engaged consulting resources. $19K ant script was result
Senior leader driving change left and new leader with different priorities took over
Within a year project team was no longer using continuous integration or maintaining unit tests
Progress is not a straight line
People from the C3 team spread out their knowledge, liking spreading seeds
Other teams start to pick up aspects of continuous integration
Continuous integration recreated to successfully deliver next version
Stories and the processes spreadlike we are doing today
Continious integration for brownfield projects
More normal situation is trying to inject continuous integration into existing process or project, rather than greenfield
Goal is to look for simple defensible approaches to move towards the goal
No cookie cutter solutions, rather look for opportunities, points of leverage, allies, problems that CI would help with
Use skunkworks if necessary, get creative
Behaviour modification example
Team with 20 plus developers, half contract Large rewrite project, used existing build
processes with addition of Cruisecontrol Build process typically was an hour plus, with
no unit tests Integrations typically took weeks, build was
typically broken for days
The solution
Lava lamps Optimizing integration build to
match developer build Impact of Change
Dropped build times to 20 minutes Eliminated build problems Broken builds became rare
occurrences
Your turn
Break up into small groups for case study Ten minutes to talk about tactics or approaches
to move towards integration Five minutes sharing results Case study is intended to give you only the
surface details. Feel free to ask questions for additional details
Concluding Remarks
Agile and Continuous are people centric so success depends on them. Seek out allies!
Don't rely on management to make things happen or enforce team norms
Skunkworks if necessary What you measure counts so make visible to the team
and management what you want changed Tools are necessary but won't determine success.
Hockey stick is required to play but doesn't mean your team will win
Failure leads to success. Progress is never linear
In conclusion
The significant problems we face cannot be solved at the same level of thinking we were at when we created them
Things should be as simple as possible, but not simpler
I am neither especially clever nor especially gifted. I am only very, very curious
Albert Einstein