Date post: | 07-May-2015 |
Category: |
Technology |
Upload: | asyncio |
View: | 1,069 times |
Download: | 0 times |
Building a Cauldron for Chef to Cook In
Jonathan Altman@async_io
For DevOpsDC August 2013
Wednesday, August 14, 2013
Background
• 12 years architecting at a leading SaaS software provider
• Started 2 companies in August 2012
• One is a SaaS/product based company
• One is a services firm
• Customers have access to their source
• No time to waste: automation good
Wednesday, August 14, 2013
What is a Cauldron?• A complete (as you want) code/artifact management and build/deployment system
• Chef server
• Git server
• Automated build system e.g. Jenkins (eventually)
• JIRA server?
• Group chat server? (IRC or XMPP)
• With a real cert
Wednesday, August 14, 2013
But SaaS is Awesome!
• Yes. Yes it is.
• Github
• Hosted Chef
• ShiningPanda, CloudBees, JenkinsHosting, travis-ci.com
• Jira OnDemand
• HipChat (or https://grove.io/ for hosted IRC, or Google+ hangouts)
Wednesday, August 14, 2013
The 3 R’s
• Redundancy
• Resiliency
• Revelation
Wednesday, August 14, 2013
Redundancy
• Does a good backup strategy ever keep just one copy of something?
• Disaster Recovery: how do you recover with your backups?
• At least for git, chef, and CI you can use multiple servers
• git “just works” in this model
• The knife plugin has backup/restore capability for example
• Not sure how to integrate with Opscode-hosted chef
Wednesday, August 14, 2013
Resiliency
• Tradeoff between your ability to deliver uptime and their motivation to address their issues that affect your uptime
• Honeypot: tradeoff again. github and hosted chef server are well protected, but a big, attractive attack vector. Your server, not so much; but is it secure?
• Ability to assign the amount of compute resources you need to deliver the performance, uptime, and redundancy you want
Wednesday, August 14, 2013
Revelation
• You will have the tools and ability to help yourself
• Git != Github, for example
Wednesday, August 14, 2013
A Cauldron: workstation
• Client workstation with
• omnibus installer (http://www.opscode.com/chef/install/)
• knife-server plugin (http://fnichol.github.io/knife-server/)
• git client software installed on it
• You could do this on your workstation’s host OS, but using a VM simplifies/isolates having multiple cauldrons
Wednesday, August 14, 2013
Create Cauldron WorkstationCocytus:vagrant_servers jonathan$ mkdir cauldron_wsCocytus:vagrant_servers jonathan$ cd cauldron_wsCocytus:cauldron_ws jonathan$ vagrant initA `Vagrantfile` has been placed in this directory. You are nowready to `vagrant up` your first virtual environment! Please readthe comments in the Vagrantfile as well as documentation on`vagrantup.com` for more information on using Vagrant.Cocytus:cauldron_ws jonathan$
Wednesday, August 14, 2013
Bootstrap the VagrantCocytus:cauldron_ws jonathan$ vi Vagrantfile # Set up your config hereCocytus:cauldron_ws jonathan$ vagrant up[default] Box base was not found. Fetching box from specified URL...[vagrant] Downloading with Vagrant::Downloaders::HTTP...[vagrant] Extracting box...[vagrant] Verifying box...[vagrant] Cleaning up downloaded box...[default] Importing base box 'base'...{bunch of lines deleted ...}[default] Booting VM...[default] Waiting for VM to boot. This can take a few minutes.[default] VM booted and ready for use!VM must be created before running this command. Run `vagrant up` first.Cocytus:cauldron_ws jonathan$
Wednesday, August 14, 2013
Install chef omnibusCocytus:cauldron_ws jonathan$ vagrant sshWelcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
* Documentation: https://help.ubuntu.com/Welcome to your Vagrant-built virtual machine.Last login: Fri Sep 14 06:23:18 2012 from 10.0.2.2vagrant@precise64:~$ sudo apt-get install build-essential curl[...bunch of output deleted]vagrant@precise64:~$ curl -L https://www.opscode.com/chef/install.sh | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 6790 100 6790 0 0 22442 0 --:--:-- --:--:-- --:--:-- 36902Downloading Chef for ubuntu...Installing ChefSelecting previously unselected package chef.(Reading database ... 51127 files and directories currently installed.)Unpacking chef (from .../tmp.8PmNsIUQ/chef__amd64.deb) ...Setting up chef (11.6.0-1.ubuntu.12.04) ...Thank you for installing Chef!vagrant@precise64:~$
Wednesday, August 14, 2013
Knife-server plugin
• knife plugin to create chef servers on the command line running on ubuntu:
• on ec2 (requires the knife-ec2 plugin as well)
• on linode (requires the knife-linode plugin)
• on an already-running instance (“standalone”)
Wednesday, August 14, 2013
Install knife-server Pluginvagrant@precise64:~$ sudo /opt/chef/embedded/bin/gem install knife-server --no-ri --no-rdocBuilding native extensions. This could take a while...Fetching: ruby-hmac-0.4.0.gem (100%)Fetching: fog-1.14.0.gem (100%)Fetching: knife-server-1.1.0.gem (100%)Successfully installed nokogiri-1.6.0Successfully installed ruby-hmac-0.4.0Successfully installed fog-1.14.0Successfully installed knife-server-1.1.04 gems installedvagrant@precise64:~$
Wednesday, August 14, 2013
Make a Server (Standalone)
• Standalone there is a bunch of other stuff to get right:
• Better if the server you’re installing on has root with ssh authorized_keys set up to your workstation’s ssh private key
• Need both private and public part of key on workstation because of http://tickets.opscode.com/browse/CHEF-4180
Wednesday, August 14, 2013
Server Bootstrapping...knife server bootstrap standalone --node-name cauldron.your.domain --host xxx.yyy.zzz.aaa[hundreds of lines of output deleted ...]192.241.179.65 Recipe: chef-server::erchef192.241.179.65 * service[erchef] action restart192.241.179.65192.241.179.65 - restart service service[erchef]192.241.179.65192.241.179.65192.241.179.65 Chef Client finished, 244 resources updated192.241.179.65 chef-server Reconfigured!192.241.179.65 Server reconfigured192.241.179.65 -----> Bootstrapping Chef Server on cauldron.async.io is complete.
If you want the web UI enabled, include --web-ui-enableWednesday, August 14, 2013
Voila?
Wednesday, August 14, 2013
uhh...Voila!
Wednesday, August 14, 2013
If You Used --web-ui-enable
BTW, login right now and change the password! Or: turn off web-uiWednesday, August 14, 2013
git server
• Several (sane) choices for hosting your own git:
• Gitolite: http://gitolite.com/gitolite/
• Gitlab: http://gitlab.org/
• Gitblit: https://code.google.com/p/gitblit/
• gitweb: https://git.wiki.kernel.org/index.php/Gitweb (please don’t)
• On windows or OSX, hosted github ($$)
• We are going to use gitolite
Wednesday, August 14, 2013
Gitolite
• Gitlab is cool, tries to reproduce as much of github other web-based git hosting as possible. Installation? Several pages of hand-invoking
• Gitblit: same goal as gitlab, but built in java. So: easy install, but big and not using the official git binaries so compatibility?
• Gitosis is dead if you run across it
• Gitweb: just don’t
Wednesday, August 14, 2013
Install
• Put the id_rsa.pub key of the user you want to admin gitolite as on the cauldron server, but name it username.pub where username is the username you want to be known as on the gitolite server
# get the softwaregit clone git://github.com/sitaramc/gitolite
# install itgitolite/install -ln
# setup the initial repos with your keygitolite setup -pk your-name.pub
Wednesday, August 14, 2013
Configure gitolite
• Back on your workstation machine:
git clone git@host:gitolite-admin.git
• Add your git repository setups in the cloned conf/gitolite.conf file
• Example, jonathan is the owner of the heatNode repository (and there is a jonathan.pub RSA public key):
repo heatNode RW+ = jonathan
Wednesday, August 14, 2013
You have a cauldron
• You can git add remote your cauldron plus any other git servers such as github to any git repository you have
git remote add origin [email protected]:name_of_your_repo.git
git remote add github [email protected]:name_of_your_repo.git
• The name after git remote add is arbitrary! “origin” is a convention but not required
• You git push/pull from all external servers so you have redundant copies
• Your cauldron is now a chef server
Wednesday, August 14, 2013
Put a real cert on the box
• As of omnibus Chef 11, webserver is nginx. Edit the ssl config to put a real server on to get rid of the big red untrusted cert warning
• Check out http://chr4.org/blog/2013/08/01/howto-use-chef-with-ssl/ for steps on how to do it with chef
• Probably a good idea, as chef-ctl-reconfigure might blast manual changes
Wednesday, August 14, 2013
Thank you. Questions?Also, thanks to @nathenharvey and @devopsdc for letting me present, @devopsdc and @fnichol (Fletcher Nichol) for the awesome real-time interactive improvements
to my presentation, and @fnichol for the awesome knife-server plugin!
Wednesday, August 14, 2013