Building a loosely coupled toolchain with Rundeck and Puppet

Outline Goal A Problem to Solve Tools Practical Examples

Building a loosely coupled toolchain withRundeck and Puppet

Steven Meunier

21 June 2014


1 GoalWhat are we trying to achieve?InfluencesInfrastructure

2 A Problem to Solve

3 ToolsWhat do we need

Something to trigger a changeSomewhere to store the data for the changeSomething to perform the change

Helping the tools help us

4 Practical ExamplesGeneric Deployment FlowUpdating Hiera DataRundeck


GoalTo have a loosely coupled deployment toolchain that leveragesour desire to manage the servers based on state

Implies decisions have already been made regarding kind oftooling and server management:

Tools that perform a particular function and do it wellTools can be swapped out if neededConcept of state can be extended to deployments if viewedas a process of changing the server state


Influences

Deployment models from gluInfrastructure as data from ansiblePuppet server statehttp://dev2ops.org/2012/05/using-rundeck-and-chef-to-build-devops-toolchains-at-chefcon/


Normal Puppet Infrastructure

Puppet Infrastructure


Deployment Infrastructure Built On Top Of Puppet

Deployment Infrastructure


Ready . . . Set . . .

By tableatny (BXP135671) [CC-BY-2.0], Source:http://upload.wikimedia.org/wikipedia/commons/1/13/Athlete_at_starting_block.jpg


Go!

By David A. Brandenburg, U.S. Navy (http://www.navy.mil/; VIRIN: 100517-N-7643B-048) [Public domain], Source:http://upload.wikimedia.org/wikipedia/commons/2/2a/Sack_race.jpg


Know your Problem Domain

DeadlinesRestrictions imposed by Security TeamTeam requirements

Release management (easy interface, keep track ofversions, support all our applications and release process)Development (APIs for jenkins, low (preferably no) adoptionbarrier for development)Operations (operations maintain control, integrate withpuppet, mcollective)


What do we need?

Something to trigger a changeSomewhere to store the information for the changeSomething to perform the change


Something to trigger a changeRundeck

Rundeck is an automation consoleMakes use of existing puppet infrastructureCan automate whatever we want, not just deploymentsFine grained access controlsEasily extensible through plugins — even if you don’t knowJavaDeployments triggered manually or via Jenkins


Somewhere to store the data for the changeHiera data

“Infrastructure as code” → “Infrastructure as data”Hiera data for configuration using Puppet 3’s data bindingChange configuration without editing puppet manifests ortemplates

property_files::parent_prefix: /srv/tomcat6/foo/confproperty_files::property_files:- foo.properties- bar.properties

properties_foo.properties:database.name: fooDBtimeout: 100


Something to perform the changePuppet & MCollective

Puppet manages all aspects of the server — nothing ismanualMcollective is an orchestration tool using activemq totrigger agents across serversMCollective manage non-state managed aspects ortemporary state changes

starting/stopping queuesdatabase deployments


No matter what you think you’ve got...

Spaceballs, c©1987 MGM Studios, Source:http://s3.amazonaws.com/mgm-assets/assets/Image/1266267325/41135/SPACEBAL-still-003-x.jpg


Sometimes your princess is a hairy guy in a dress

Spaceballs, c©1987 MGM Studios, Source:http://img.pandawhale.com/65424-Spaceballs-Celebrity-stunt-dou-DZzt.jpeg


Tools need help to make them fit

Git: steep learning curveRundeck: needs extra plugins to fit your workflowPuppet: duplicate declarations; ordering; limitations ofhieraMCollective: can be unreliable if used as a source of nodetruth; time needed to build the agentsRuby: it’s critical if you’re using puppet + mcollective


Generic Deployment Flow

Create deployment window with no puppet interferenceBuild rpms from deployable unit (if applicable)Disable monitoringCompare and update configuration (if applicable)Update version parameter in hiera dataKick off puppet runRestart serviceRe-enable monitoring


Deployment Configuration Committer

Ruby script with the goal of reducing manual actionsrelated to configurationWorks with java properties files and context xml filesBuilds a hash of properties in the release and compares itagainst the current propertiesUploads any changes to Gerrit for review


An example tomcat application in Hiera

tomcat_context:ROOT:

version: xx.yy.zz

tomcat_instance:foo: {}

tomcat_context:ROOT:

artifact: fooartifact_provider: fileartifact_target: ROOT.warartifact_url: http//repo.example.com/warinstance: foo

rundeck_job:foo: {}


Puppet Rundeck Job Type

@@rundeck_job { "${project}/${group}/${name}_${hostname}":ensure => $ensure,jobname => $name,project => $project,group => $group,definition => template($template),

}


Rundeck Conditional Local Command

#!/bin/shCOMMAND_TRUE=$1COMMAND_FALSE=$2CONDITION="[ $3 ]"

EXIT_CODE=0

if [ "x$COMMAND_FALSE" == "x" ]then

if eval $CONDITIONthen

$COMMAND_TRUEEXIT_CODE=$?

fielse

if eval $CONDITIONthen

$COMMAND_TRUEEXIT_CODE=$?

else$COMMAND_FALSEEXIT_CODE=$?

fifi

exit $EXIT_CODE


Rundeck Conditional Local Command


Thank You

Date post:	10-May-2015
Category:	Engineering
Upload:	smeunier114
View:	1,243 times
Download:	2 times

Building a loosely coupled toolchain with Rundeck and Puppet

Engineering