Puppet @Twitter

Andrew Hamilton

@ahamilton55

Who am I?

What we’ll cover

The Dev in DevOps

Version Control

Peer review of code

Review Boardhttp://www.reviewboard.org/

Define your style

OWNERS

OWNERS example

Moving from Puppet 2.7 to Puppet 3.x

Need to go from:

to:source => ‘puppet:///<module>/<filename>’

source => ‘puppet:///modules/<module>/<filename>’

But of course…

You can automate all of this!

Review Board – git review tools

Style Guide – puppet-lint

OWNERS – Script a hook and git review tools

Version control hooks

And always iterate on this

Branching

Head

Our true testing branch

Where development happens

Systems don’t live on this branch

Testing

Works along with our canary system for deployments

Major changes can sit here for a couple days to weeks

Production

99% of our systems are on the production branch

Testing on puppet and head should prevent any error from propagating to production

Workflow

Head

Develop

Review Board

Commit

Test

Testing Production

Cherry pick

Review Board

Test

Commit

Cherry pick

Review Board

Commit

But it could be better

puppetmasterd

Each host runs 3 puppetmasterd processes

Each can serve between ~1k – 2k hosts

We use round-robin DNS

ENC

Audubon

Facts

Mostly deprecated, legacy

Pulled info from Facter

Attributes

Basic host information

Two levels - server, role

Allows for inheritance

Groups

Allows for a logical grouping of hosts

Tells you things like role, rack, cluster, etc

Pulling in info from the ENC is easy

We have functions that pull in each type of change in a manifest

attribute(‘<attr_name>’, ‘<default_value>’)

group(‘<group_name>’, ‘<default_value>’)

Our ENC data store is also what controls the puppet branch our hosts use

We set a group called puppet_branch which is used by our client cronjob

Agents

1. Find branch the host is using

2. Find the port that branch uses on masters

3. Get a master from DNS

1. Health check the master

2. If health continue, else repeat until timeout

4. Run puppet apply against the master

and port

Modules

Team Modules

Base Modules

Service Modules

Create a module for each system service

Chain these together in your service modules

If you do something often, create a module that every group can use

Build for self-service

Looking forward