Post on 27-Aug-2014
description
transcript
Streamlining your puppet development workflow
Tomas Doran @bobtfish 2014-‐07-‐07
Why invest in your workflow?
2
More time
3
To concentrate on
4
The important things in life
5
Like dwarf fortress!
6
Why invest in your workflow?• Productivity!
• Work smarter, not harder
• You spend a lot of time writing/testing/debugging code • Optimizing that is worthwhile
• Faster development cycle is more productive • 5% faster cycle • 5% more time for testing • Less bugs
7
Revision control
• You must have your code in revision control
• git is preferred • fast + cheap branches • everyone else uses it • github • gitolite
!• svn is also ‘workable’
8
How do you run puppet?
• I like cron (daemon also fine) • Two possible approaches • —noop mode automatically + manual apply • Automatic apply
9
How do you run puppet?
• I like cron (daemon also fine) • Two possible approaches • —noop mode automatically + manual apply • Automatic apply
10
How do you run puppet?
• I like cron (daemon also fine) • Two possible approaches • —noop mode automatically + manual apply • Automatic apply
• I recommend automatic apply • Scary (don’t push to master unless you’re
confident!) • puppet agent —disable (monitor this!) • Testing workflow • Eventual consistency
11
Move just fast enough to not break everything
• Test so that you’re confident • Branch for every significant change • Reduce batch size • Small scary change easier to test • Easier to roll back
• Otherwise - applying months of changes at once • Really scary! • Don’t even know desired effects!
• Communicate!
12
Standalone modepuppet apply --modulepath modules/:vendor/modules/ --show_diff --hiera_config=./standalone-hiera.yaml $@ manifests/site-standalone.pp !
13
—noop
• Use —noop mode for testing!
ssh -A "$HOST" -- "sh -c 'cd $DESTDIR/$PUPPET_DIRNAME; ./tools/puppet-standalone --verbose --show_diff —noop'"
• tools/what-would-happen-on
14
Dynamic environmentsgit branch => puppet environment
puppet agent -t —environment my_test_branch !
• puppet >= 2.7 has environment support
• Use puppetupdate or r10k to push branches (Links at the end!)
15
16
Reporting
• Need to know what puppet did • Puppet has logs + reporting functionality • Push reports to: • irc • email (eww!) • elasticsearch • mysql • puppetdb
• Saves compiled catalogs to disk tools/what-just-happened-on
17
Foreman
18
Norman
19
Puppet enterprise
20
puppet-syntax
• Ruby gem • Trivial to add to your project • Checks .pp, .erb, .yaml • Fast enough to run pre-commit
echo ‘bundle exec rake syntax’ \ >.git/hooks/pre-commit \ chmod 755 .git/hooks/pre-commit
21
r10k/librarian - Puppetfile• Awesome module deployment - with robots! • Easy vendor/modules directory for modules from the forge • Makes module = git repository pattern easier • Not every module from the forge is useable immediately
• Fork on github (and make your changes open source)? • Pull request and get them back upstream! • Fork into internal git and modify.
• gitolite mirrors • Improve performance • No external dependencies
• Private forge (puppet-library)22
Module template
• ‘puppet module generate’ uses a template • Start from the GDS example one:
github.com/gds-operations/puppet-module-skeleton • Modify to your taste!
23
‘Real’ testing
• No hard rules. • Invest to the level that’s right for you!
• Dev heavy teams • Know about unit testing!
• Sysadmin heavy teams • Less enthusiastic
• Do what provides value!24
Feedback!
• Tighten your OODA loop! • Don’t care how! • N.B. Automated tests don’t work unless they’re
automated. • I.E. MUST run on commit
• Whatever’s effective for your org25
rspec-puppet
• Unit testing • At least write a compile test for your code! • Put it in your module template. • Explicit dependencies FTW
!• Use puppetlabs-spec-helper • Inject mocks into spec/fixtures/manifests/site.pp
echo ‘define my::complex::dependency ($foo, $bar) {}’ >> spec/fixtures/manifests/site.pp
26
serverspec
• Spin up Vagrant VM and apply your code • Check properties of: • files • ports • services
• Acceptance testing • Slower and heavier weight than unit tests • Can be highly valuable!
27
Jenkins - simple• Put your tests together so that they can be run as one job
!rake test
task :test => [:syntax, :spec, :integration]
• Get Jenkins to run it on commit to master • git polling • + add a post-receive hook to curl Jenkins
• Shout in email + irc!28
Jenkins - less simple
• If branches are cheap (i.e. git!) • Encourages people to push branches • Code review++ • Adhoc • Or pick your poison
• Run syntax checks and unit and/or integration tests on every branch.
• Report back to committer • irc notification • write in code review
29
Jenkins integration branches• Jenkins can merge branches! • Push a branch • Jenkins picks it up • Merges with master • Runs tests • If they pass, pushes results
30
Jenkins integration branches• Jenkins can merge branches! • Push a branch • Jenkins picks it up • Merges with master • Runs tests • If they pass, pushes results
31
Do everything in irc
32
True believers
33
34
Policy vs Automation
• Testing is awesome • Code review is awesome • ‘Process is the scar tissue from previous problems’ • Empower people to change the process!
• If you make the tools simple to use… • People will use them! • Make doing the right thing a no-brainer
35
We’re hiring! NY, SF, Dublin, London! !
• https://slideshare.net/bobtfish/ • http://puppetlabs.com/blog/git-workflow-and-puppet-environments • http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-1/ • http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/ • https://github.com/youdevise/puppetupdate • https://github.com/adrienthebo/r10k • http://gitolite.com/gitolite/index.html • http://puppetlabs.com/puppet/puppet-enterprise/ • http://docs.puppetlabs.com/guides/reporting.html • http://theforeman.org/ • https://github.com/youdevise/norman/ • https://github.com/youdevise/puppet-logstash-reporter/ • https://github.com/gds-operations/puppet-module-skeleton • https://github.com/drrb/puppet-library 36