Post on 29-Jun-2015
description
transcript
Testing Driven Puppet Modules[Nan Liu @sesshin]
Copyright 2014
1 / 45
Overview
2 / 45
Overview
Who
This talk is not for people who:
Write perfect code (manifests)Never upgrade PuppetDon't have SLAUsed Puppet since 0.2x
3 / 45
Overview
Who
Why
Testing Puppet on Production is __
4 / 45
Overview
Who
Why
Positive feedback loop:
5 / 45
Overview
Who
Why
Creating a virtuous cycle:
6 / 45
Overview
Who
Why
Creating a virtuous cycle:
7 / 45
Overview
Who
Why
What
Developing Puppet Modules
Development feedbackpuppet-lintpuppet-syntaxrspec-puppet
TestingPackerVagrantbeaker
8 / 45
Development Feedback
9 / 45
Development
puppet-lint
Manifest lint check:
$ puppet lint manifests/**/*WARNING: top-scope variable being used without an explicit namespace on line 4WARNING: class inheriting from params class on line 26WARNING: class not documented on line 1WARNING: line has more than 80 characters on line 52
10 / 45
Development
puppet-lint
Manifest lint cleanup:
$ puppet-lint -f demo.ppFIXED: string containing only a variable on line 1FIXED: variable not enclosed in {} on line 5FIXED: indentation of => is not properly aligned on line 2FIXED: indentation of => is not properly aligned on line 3FIXED: indentation of => is not properly aligned on line 5WARNING: ensure found on line but it's not the first attribute on line 4
11 / 45
Development
puppet-lint
Configure puppet-lint behavior
PuppetLint.configuration.disable_80charsPuppetLint.configuration.disable_arrow_alignmentPuppetLint.configuration.disable_class_inherits_from_params_classPuppetLint.configuration.disable_class_parameter_defaultsPuppetLint.configuration.fail_on_warnings = true
PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp']
12 / 45
Development
puppet-lint
puppet-syntax
Manifest validation:
$ puppet parser validate manifests/bad.ppError: Could not parse for environment production: Syntax error at 'demo';expected '}' at /Users/nan/src/puppet-demo/manifests/bad.pp:4
13 / 45
Development
puppet-lint
puppet-syntax
Rakefile
require 'puppet-syntax/tasks/puppet-syntax'PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"]PuppetSyntax.future_parser = true
14 / 45
Development
puppet-lint
puppet-syntax
Puppet module folder:
manifests/tests/Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
PuppetLint.configuration.disable_80charsPuppetLint.configuration.disable_arrow_alignmentPuppetLint.configuration.disable_class_inherits_from_params_classPuppetLint.configuration.disable_class_parameter_defaultsPuppetLint.configuration.fail_on_warnings = true
PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp']
require 'puppet-syntax/tasks/puppet-syntax'PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"]PuppetSyntax.future_parser = true
15 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
RSpec-Puppet
Compile catalogmodule dependencysystem factclass parameters
Verifies catalogClass/Resource specificationRelationshipsExpectation
16 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
.fixtures.yml
fixtures: repositories: apt: "https://github.com/puppetlabs/puppetlabs-apt.git" stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" firewall: "https://github.com/puppetlabs/puppetlabs-firewall.git" concat: "https://github.com/puppetlabs/puppetlabs-concat.git" symlinks: postgresql: "#{source_dir}"
17 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Supply system facts and class parameters:
let(:facts) {{ :osfamily => 'redhat' }}let(:params) {{ :keys_enable => true, :keys_file => '/etc/ntp/ntp.keys', :keys_trusted => ['1', '2', '3'], :keys_controlkey => '2', :keys_requestkey => '3',}}
18 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
require 'spec_helper'
describe 'ntp' do Dir.glob('tests/*.pp').each do |file| let(:facts) {{ :osfamily => 'redhat' }} context file do let(:pre_condition) { File.read(file) } it{ should compile } end endend
19 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Puppet Class:
describe 'ntp' do let(:facts) {{ :osfamily => 'RedHat' }} let(:params) {{ :iburst_enable => true, }}
it do should contain_file('/etc/ntp.conf').with({ 'content' => /iburst\n/, }) endend
20 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Define Type
describe 'mysql::db', :type => :define do let(:facts) {{ :osfamily => 'RedHat' }} let(:title) { 'test_db' }
let(:params) { { 'user' => 'testuser', 'password' => 'testpass', } }...end
21 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Puppet module folder:
.fixtures.ymlRakefilemanifests/tests/spec/classes/*spec/defines/*
22 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Gemfile
source "https://rubygems.org"
group :test do gem "rake" gem "puppet", ENV['GEM_PUPPET_VERSION'] gem "puppet-lint" gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git' gem "puppet-syntax" gem "puppetlabs_spec_helper"end
23 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
How many modules do you have?
modulesync
24 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
Gem::Specification.new do |s| ... if facter_version = ENV['GEM_FACTER_VERSION'] s.add_runtime_dependency 'facter', facter_version else s.add_runtime_dependency 'facter' end
if puppet_version = ENV['GEM_PUPPET_VERSION'] || ENV['PUPPET_GEM_VERSION'] s.add_runtime_dependency 'puppet', puppet_version else s.add_runtime_dependency 'puppet' end
s.add_runtime_dependency 'rake' s.add_runtime_dependency 'rspec', '~> 2.11.0' s.add_runtime_dependency 'mocha', '~> 0.10.5' s.add_runtime_dependency 'puppetlabs_spec_helper', '0.7' s.add_runtime_dependency 'rspec-puppet' s.add_runtime_dependency 'puppet-lint', '~> 1.0' s.add_runtime_dependency 'puppet-syntax'
s.files = Dir.glob('lib/**/*') + %w(LICENSE) s.require_path = 'lib'end
25 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
Gemfile
group :development, :test do gem 'bodeco_module_helper', :git => 'https://github.com/bodeco/bodeco_module_helper.git'end
Rakefile
require 'bodeco_module_helper/rake_tasks'
26 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
$ rake -Trake beaker # Run beaker acceptance testsrake beaker_nodes # List available beaker nodesetsrake build # Build puppet module packagerake clean # Clean a built module packagerake coverage # Generate code coverage inform...rake help # Display the list of available...rake lint # Check puppet manifests with p...rake spec # Run spec tests in a clean fix...rake spec_clean # Clean up the fixtures directoryrake spec_prep # Create the fixtures directoryrake spec_standalone # Run spec tests on an existing...rake syntax # Syntax check Puppet manifests...rake syntax:hiera # Syntax check Hiera config filesrake syntax:manifests # Syntax check Puppet manifestsrake syntax:templates # Syntax check Puppet templatesrake validate # Check syntax of Ruby files an...
27 / 45
Development
puppet-lint
puppet-syntax
rspec-puppet
Summary
puppet-lint: style enforcerpuppet-syntax: code parserspec-puppet: catalog verifypin your versions
28 / 45
Testing
29 / 45
Testing
Packer
Say no to mystery boxesVirtualbox or VMware fusion/workstationAmazon, Digital Ocean, GCE, ...
30 / 45
Testing
Packer
"builders": [{ "vm_name": "centos70", "type": "vmware-iso", "guest_os_type": "centos-64", "http_directory": "http", "iso_url": "{{ user iso_url }}", "iso_checksum": "{{ user iso_checksum }}", "tools_upload_flavor": "linux", "boot_command": [ "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort}}/ks7.cfg<enter>" ], "disk_size": 10140, "vmx_data": { "memsize": "512", "numvcpus": "1", "cpuid.coresPerSocket": "1" } }]
31 / 45
Testing
Packer
"provisioners": [{ "type": "shell", "environment_vars": [ "CM={{user cm}}", "CM_VERSION={{user cm_version}}", "CM_SET_PATH={{user cm_set_path}}", "CLEANUP_PAUSE={{user cleanup_pause}}" ], "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S bash '{{.Path}}'", "scripts": [ "script/fix-slow-dns.sh", "script/sshd.sh", "script/vagrant.sh", "script/vmtool.sh", "script/cmtool.sh", "script/cleanup.sh" ] }],
32 / 45
Testing
Packer
https://github.com/puppetlabs/puppet-vagrant-boxeshttps://github.com/mitchellh/veewee-to-packerhttps://github.com/box-cutterhttps://github.com/hashicorp/puppet-bootstrap
33 / 45
Testing
Packer
Vagrant
Vagrant.configure('2') do |conf| conf.vm.define 'demo' do |mod| mod.vm.box = 'centos64.box'
mod.vm.synced_folder './modules', "/opt/puppet/share/puppet/modules" mod.vm.synced_folder './manifests', "/etc/puppetlabs/puppet/manifests" mod.vm.synced_folder './data', "/etc/puppetlabs/puppet/data"
mod.vm.provision :puppet do |p| p.module_path = 'spec/fixtures/modules' p.manifests_path = 'manifests' p.manifest_file = ENV['VAGRANT_MANIFEST'] || 'init.pp' p.options = '--verbose' end endend
34 / 45
Testing
Packer
Vagrant
https://github.com/adrienthebo/vagrant-config_builder
$ vagrant plugin install vagrant-config_builder
├── config│ ├── roles.yaml│ └── vms.yaml└── Vagrantfile
---vms: - name: db private_networks: [ {ip: '10.20.1.2'} ] box: centos-5-i386 hostname: db.puppetlabs.vm synced_folders: - host_path: '.' guest_path: '/vagrant' disabled: true provisioners: - type: puppet manifests_path: 'tests' module_path: 'spec/fixtures/modules' manifest_file: <%= ENV['VAGRANT_MANIFEST'] || 'init.pp' %>
35 / 45
Testing
Packer
Vagrant
def vm(opt) module_name = opt.fetch(:module).to_s || raise(ArgumentError, 'Must provide puppet module name' hostname = opt.fetch(:hostname, module_name).to_s memory = opt.fetch(:memory, 512) cpu = opt.fetch(:cpu, 1) os_type = opt.fetch(:type, :linux)
Vagrant.configure('2') do |conf|
conf.vm.network(:forwarded_port, guest: port, host: port, auto_correct: true) if port
if os_type == :windows conf.ssh.username = 'vagrant' conf.winrm.username = 'vagrant' conf.winrm.password = 'vagrant' end...
36 / 45
Testing
Packer
Vagrant
vm( :hostname => 'oel', :module => 'application', :memory => 8096, :box => 'oracle65-pe3.2.3', :port => 8080)
37 / 45
Testing
Packer
Vagrant
$ vagrant up$ vagrant provision$ vagrant ssh$ vagrant destroy
38 / 45
Testing
Packer
Vagrant
Beaker
Puppet Labs testing framework
spec/acceptance├── class_spec.rb├── disable_monitoring_spec.rb├── nodesets│ ├── centos-59-x64.yml│ ├── centos-64-x64-pe.yml│ ├── centos-64-x64.yml│ ├── centos-65-x64.yml│ ├── default.yml│ ├── fedora-18-x64.yml│ ├── sles-11-x64.yml│ ├── ubuntu-server-10044-x64.yml│ ├── ubuntu-server-12042-x64.yml│ └── ubuntu-server-1404-x64.yml├── ntp_config_spec.rb├── ntp_install_spec.rb├── ntp_parameters_spec.rb├── ntp_service_spec.rb├── preferred_servers_spec.rb├── restrict_spec.rb└── unsupported_spec.rb
39 / 45
Testing
Packer
Vagrant
Beaker
HOSTS: centos-64-x64: roles: - master platform: el-6-x86_64 box : centos-64-x64-vbox4210-nocm box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box hypervisor : vagrantCONFIG: type: foss
40 / 45
Testing
Packer
Vagrant
Beaker
it 'should run successfully' do pp = "class { 'ntp': }"
# Apply twice to ensure no errors the second time. apply_manifest(pp, :catch_failures => true) do |r| expect(r.stderr).not_to match(/error/i) end apply_manifest(pp, :catch_failures => true) do |r| expect(r.stderr).not_to eq(/error/i)
expect(r.exit_code).to be_zero endend
41 / 45
Testing
Packer
Vagrant
Beaker
it 'starts the service' do pp = <<-EOS class { 'ntp': service_enable => true, service_ensure => running, service_manage => true, service_name => '#{servicename}' } EOS apply_manifest(pp, :catch_failures => true)end
describe service(servicename) do it { should be_running } it { should be_enabled }end
42 / 45
Testing
Packer
Vagrant
Beaker
Summary
Packer: VM buildVagrant: VM clone and testingBeaker: Automated testing
43 / 45
Questions
44 / 45
Thank You![nan@bodeco.io]
45 / 45