Introduction to orchestration using

MCollective

Pieter Loubser!Puppet Camp London!

18 November 2013

Who am I?

Developer at Puppet Labs in London!

On the MCollective team for 2 years!

@pieterloubser on the twitter!

ploubser on IRC

Finite State Machines

Abstract machine that can only be in one of a finite number of states!

It can move from one state to another when an event or condition is triggered (transitions)

Old app Magic? Winning

Orchestration It can be a Finite State Machine

Determine your states!

Determine your transitions

What is MCollective?!

A framework for building server orchestration tools and a parallel job execution system

The moving parts

Message Oriented Middleware!

MCollective Server (mcollectived)!

MCollective Client

Middleware

Client

RPC Request

Server Server Server

RPC Request

RPC RequestRPC Request

Server

Request : Determine the status of the puppet service

Response

Agents

Actionsrpcutilpuppet

packageservice

...

statusstartstop...

Middleware

Client

Server Server Server

stopped stopped running

Process results

Agents - State Transitions

Agents can change the state the world is in!

Agents can determine what state the world is in !

An agent consists of two parts!

The DDL file which describes the Agent

Agents - State Transitions

Agents can change the state the world is in!

Agents can determine what state the world is in !

An agent consists of two parts!

The DDL file which describes the Agent!

An implementation file that defines the actions

State We’re not always looking at the system as

a whole

MCollective uses discovery plugins as the source of truth!

But touching all the things can sometimes be a bad idea!

There is a desire for fine grained filtering

Filters

Filter on fact sources

Filters

Filter on fact sources!

Filter on Puppet classes

Filters

Filter on fact sources!

Filter on Puppet classes!

Filter on identity

Filters

Filter on fact sources!

Filter on Puppet classes!

Filter on identity!

Complex filtering based on logical language

Data Plugins

Enables you to determine system state by executing code in a similar manner to agents!

Consist of the same two parts as Agents!

DDL!

Implementation

Orchestration A simple deployment scenario

5 web servers sitting behind a load balancer!

Mark a web server as down!

Update the application!

Mark the web server as up!

Profit

lb1

web1 web2 web3 web4 web5

Group 1

Removed from load balancer

Failed to remove from load balancer

Ready for upgrade

Failed upgrade

Successful upgrade

Failed to return to load balancer Returned to

load balancer

Pre deployment Ready forload balancer

Successful deploy

Failed deploy

The code

https://github.com/ploubser/puppetcamp-london

Here goes everything…

Questions?