Automation & the Deployment Pipeline

Adam Goucher@adamgoucher

adam@element34.caThis workshop is about the various forms of automation that happens in the Deployment Pipeline.

On this slide we talk about:- The Going Camping Heuristic

photo credit: http://www.flickr.com/photos/nataliejohnson/2763306217/

End-user PerspectiveFrom the end-user perspective, your application just delivers its <thing> to them.

photo credit: http://www.flickr.com/photos/auntlaura/4806934654/

RealityWe know that the reality is slightly different. But even this is well organized and logical -- if you understand the system.

photo credit: http://www.flickr.com/photos/incandopolis/2867941478/

An Ethical DistinctionOn this slide we talk about:- Continuous Delivery vs Continuous Deployment

http://www.flickr.com/photos/8840244@N04/3644022974/

The Lightsaber HeuristicThere are commercial tools available to do all this stuff. Instead, we’ll use all open-source tools to build a custom solution.

On this slide we talk about:- The Lightsaber Heuristic

photo credit: http://www.flickr.com/photos/pasukaru76/3694179131/

Lots of parts, working togetherTo have a fully automated deployment pipeline, there are /lots/ of moving parts. The big ones are- acceptance criteria- configuration management- continuous integration- data migration- deployment- functional test automation- immunization systems.

We will be talking about each of these

photo credit: http://www.flickr.com/photos/19779889@N00/4398186065/

The best way to illustrate all these things is to do them live. As a result, we will create a Ruby-on-Rails application and have push it through each part of the Deployment Pipeline. Both the application and all the pipeline support code will be made available shortly after the workshop.

SpecificationsIt can be argued when the Pipeline really starts. So lets start from the beginning; with acceptance criteria.

On this slide we talk about:- Specification Workshops- Given / When / Then

I’m not a fan of any of the current BDD/ATDD tools from an automation perspective, but going through the exercise of producing the specifications absolutely has merit.

photo credit: http://www.flickr.com/photos/wallyhartshorn/2680766935/

Manual InspectionFor our purposes though, the automation part begins when someone inspects their change as to whether it is time to Commit.

photo credit: http://www.flickr.com/photos/mooney77/229524092/

Commit

There is one path through the pipeline and our Commit starts it.

On this slide we talk about:- Check-in vs. Commit- Clean vs. dirty trunk- Pre-Commit checks- Commit checks- Post-Commit checks

photo credit: http://www.flickr.com/photos/jkgroove/119112238/

‘Cans’Once our code is versioned, we need to have machines to run our checks against.

On this slide we talk about:- Machines as ‘cans’

photo credit: http://www.flickr.com/photos/neonman/1438473495/

Puppet

Puppet

On this slide we talk about:- Puppet vs. Chef vs. Cfengine- What Configuration Management is.... and isn’t

And we provision our server, demonstrating how Puppet controls- packages- services- users/groups

photo credit: http://www.flickr.com/photos/sukigirl74/2716951216/

ContinuousIntegration

With a machine to run things again, we now turn our attention to the control center of the whole Pipeline; the Continuous Integration server.

photo credit: http://www.flickr.com/photos/clintjcl/2847746499/

Jenkins

Jenkins (part one)

On this slide we talk about:- Role of CI- Jenkins vs. Hudson

And we configure Jenkins for our application, demonstrating- Jobs- Job chains- Authentication- Remote agents- Reporting- Notifications- Plugins

Data MigrationData Migration is tricky. To say the least. But getting it right is critical to getting your pipeline fully automated

On this slide we talk about:- Up- Down- Idempotency

Because we’re using Ruby-on-Rails, we’ll use ActiveRecord Migrations for our pipeline but we’ll also look at dbdeploy

photo credit: http://www.flickr.com/photos/theclyde/1663944456/

DeploymentCompared to everything else before now, pushing the code out onto the server(s) is a snap.

On this slide we talk about:- Zero-downtime deploys- Rollbacks

Most ‘platforms’ have a deployment tool that has grown with it. For our deployments, we’ll be using Capistrano. Cap is not limited to just Ruby-on-Rails deployments though so we’ll talk about how to customize it for a number of different scenarios.

photo credit: http://www.flickr.com/photos/mybloodyself/464597917/

Functional AutomationSo we’re finally at the point where we can do some ‘traditional’ automation. And since our stack is all Open Source, we’re going to use Selenium.

photo credit: http://www.flickr.com/photos/kwc/1866220494

Papers please

Uniquely identifying things on the page is one of the hardest parts about doing web automation. And is one of the main reasons scripts are ‘brittle.’

On this slide we talk about:- attribute-based locators- structural locators- when to use which locator

photo credit: http://www.flickr.com/photos/michaelimage/2843233811/

Synchronization

The other common pain point for web automation is synchronizing the script steps with the application

On this slide we talk about:- Web 1.0- Web 2.0- Web 2.0++

photo credit: http://www.flickr.com/photos/bfauld1/121955292/

Randomness

Introducing randomness into your scripts is important -- use it!

On this slide we talk about:- Providers- Oracles

photo credit: http://www.flickr.com/photos/richardsummers/195510085/

Pesticide ParadoxImagine you are a farmer...

On this slide we talk about:- The Pesticide Paradox

photo credit: http://www.flickr.com/photos/gregoryjordan/4735058616/

Land mine problemYou should find the mines.

Not.

Your.

Customer.

On this slide we talk about:- The Land Mine Problem- Continuous Delivery vs. Continuous Deployment

photo credit: http://www.flickr.com/photos/d-minor/120407151/

TagsManual built suites of scripts is a code smell

On this slide we talk about:- Tags- Tag taxonomies- Script discovery- Parallel execution- Random execution

photo credit: http://www.flickr.com/photos/thingsarebetterwithaparrott/1437904057/

Page Objects

Page Objects bring the power of OO to automation

On this slide we talk about:- Page Objects

For our project, we will construct some Page Object based Selenium scripts using RSpec. Though we can discuss and/or create examples of them in other languages too.

photo credit: http://www.flickr.com/photos/dmclean/2472399095/

Jenkins (Redux)

Jenkins (again)

On this slide we talk about:- Using Jenkins as a replacement for se-grid- Advanced job chaining

Immunization Systems

Immunizations shots keep you healthy. Immunization systems keep your deployments healthy.

On this slide we talk about:- Immunization Systems- Business vs. Technical metrics- Incremental deployments- Automated rollback

In order to ensure our deployment will not negatively impact our users, we will build a basic Immunization System for our application. By their nature, these are highly customized per application.

photo credit: http://www.flickr.com/photos/irenemiranda/4683185743/

G-ForcesKent Beck uses the notion of G-Forces to explain how things respond to speeding up your process. With a fully automated deployment pipeline things will be able to go faster. And faster.

But you could also get thrown from the carousel, so be careful.

On this slide we talk about:- Kent Beck’s G-Forces

photo credit: http://www.flickr.com/photos/areyoumyrik/308908736/

Adam Goucher@adamgoucher

adam@element34.ca

photo credit: http://www.flickr.com/photos/teimoury/2431264409/