Building Reusable Modules

Jon Topper

Tuesday, 2 April 13

Jon who?★ 12 years professional Linux sysadmin experience

★ Principal Consultant, The Scale Factory

★ 4.5 years using Puppet (since 0.24)

★ 4 full-time staff + subcontractors

★ >20 customer infrastructures deployed

Tuesday, 2 April 13

Our Problem Domain★ Multiple customers

★ Multiple Linux distributions

★ Multiple application environments

★ Adding new features over time

★ Requirement to support old deployments

★ Multiple individuals contributing

Tuesday, 2 April 13

Your Problem Domain?★ Multiple teams in a large organisation

★ Forge contributions

Tuesday, 2 April 13

Our Approach

apache

stdlibfirewall

mysqlfirewallyum varnish

concat

role_web role_db role_lb

defaults

Tuesday, 2 April 13

Reusable?★ Use the same modules

★ In different combinations

★ To solve different problems

★ Without modification

Tuesday, 2 April 13

Single Responsibility

Install MySQLConfigure MySQLStart MySQL ✔Install MySQLConfigure MySQLInstall ApacheInstall mod_phpInstall WordpressConfigure MySQL grantsConfigure Wordpress ✘

Tuesday, 2 April 13

Low Coupling★ Changes in highly coupled modules necessitate

changes in others

★ Limit coupling where possible

★ Examples of coupling:

★ Using variables from another class

★ Requiring specific resources from another class

★ Any other use of non-public interfaces

Tuesday, 2 April 13

# sf_apache/manifests/init.ppclass sf_apache::service { service { “httpd”: ensure => running }}

# sf_puppetmaster/manifests/init.ppclass sf_puppetmaster::config { file { “/etc/httpd/conf.d/puppetmaster.conf”: notify => Class[“sf_apache::service”], }} ✔ file { “/etc/httpd/conf.d/puppetmaster.conf”: notify => Service[“httpd”], } ✘

Tuesday, 2 April 13

Tell, Don’t Ask

class apache::config { case $hostname { /devel/: { $port = 8080 } default: { $port = 80 } } ...} ✘

class apache::config( $port = 80 ) { ...} ✔

Tuesday, 2 April 13

Tell, Don’t Ask: Exceptionclass apache::params { case $osfamily {

'RedHat': { $conf_d_path = "/etc/httpd/conf.d" $user = 'apache' }

'Debian': { $conf_d_path = "/etc/apache2/conf.d" $user = 'www-data' } }} ✔

Tuesday, 2 April 13

Parameters★ Check user parameters for syntax

★ Choose sensible defaults

★ Document the module’s public interface

★ Write tests for that interface

Tuesday, 2 April 13

Open/Closed Principle★ Modules should be open for extension

★ But closed for modification

★ Ensures backward compatibility

★ Tests prove that compatibility

Tuesday, 2 April 13

Open/Closed Principle

define sf_firewall::basic( $proto, $dport = undef, $source = undef, $destination = undef, $action,) {

# Create a basic firewall rule # Assumes ‘INPUT’ chain

}

Tuesday, 2 April 13

Open/Closed Principle

define sf_firewall::basic( $proto, $dport = undef, $source = undef, $destination = undef, $action, $chain,) { # Create a basic firewall rule. Chain now # a required parameter with no default}

✘Tuesday, 2 April 13

Open/Closed Principle

define sf_firewall::basic( $proto, $dport = undef, $source = undef, $destination = undef, $action, $chain = ‘INPUT’) { # Create a basic firewall rule. Chain now # defaults to ‘INPUT’ but can be changed.}

✔Tuesday, 2 April 13

Automated Test Pipeline★ Smoke Tests

★ Unit Tests

★ Integration Tests

Tuesday, 2 April 13

Smoke Testing★ puppet-lint to enforce house style

★ puppet parser validate to check syntax

★ Evaluate ERB templates to check syntax

Tuesday, 2 April 13

Unit Testing: rspec-puppet★ Rapid feedback testing

★ Can easily target multiple ruby/puppet version combinations

★ Test your module’s behaviour - not Puppet’s

★ Aim for Condition Coverage across behaviour that varies on parameter

★ Doesn’t guarantee that your module will work as planned in the real world!

Tuesday, 2 April 13

Condition Coverageclass sf_firewall::params {

case $osfamily {

'RedHat': { $persist_command = "/sbin/iptables-save > /etc/sysconfig/iptables" }

'Debian': { $persist_command = "/sbin/iptables-save > /etc/iptables/rules.v4" }

}

Tuesday, 2 April 13

Condition Coveragedescribe ‘sf_firewall’, :type => ‘class’ do

def self.test_standard_behaviour ... end

context "on a RedHat OS" do let :facts do { :osfamily => 'RedHat' } end

it { should contain_exec('persist-firewall').with_command( "/sbin/iptables-save > /etc/sysconfig/iptables" ) }

test_standard_behaviour

end

Tuesday, 2 April 13

Integration Testing: Cumberbatch★ Based on Cucumber

★ Uses Vagrant to start virtual machines

★ Runs puppet manifests for real

★ Tests outcomes

★ Rolls back VM state and tries different manifests

★ Does it all again for a different OS version

★ Very slow!

Tuesday, 2 April 13

Integration Testing: CumberbatchScenario: Basic install of Apache Given there is a running VM called "server" When I apply a puppet manifest containing: """ include cucumber_defaults class { sf_apache: 'Port' => '80', 'Children' => '10' } """ Then a second manifest application should do nothing

And there should be 11 processes called “httpd” running And the Apache module “core_module” should be loaded And the Apache module “rpaf_module” should be loaded And a process called “httpd” should be listening on TCP port 80 And a GET request to http://localhost/server-status/ should return an http status of 200

Tuesday, 2 April 13

Questions?

Tuesday, 2 April 13

Jon Topper

jon@scalefactory.com

http://www.scalefactory.com/

Twitter: @jtopper

Tuesday, 2 April 13