Post on 11-Jun-2015
description
transcript
Manageable Puppetinfrastructure~September 2014 edition~
PuppetConf San Francisco
Ger Apeldoorn - http://puppetspecialist.nl
1 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Who's this?
2 / 44
Manageable Puppetinfrastructure~September 2014 edition~
PuppetConf San Francisco
Ger Apeldoorn - http://puppetspecialist.nl
1 / 44
ScopeAlso... why this talk?
3 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Who's this?
2 / 44
Commonpitfalls
4 / 44
ScopeAlso... why this talk?
3 / 44
Pitfalls
Cause & effectPitfalls
Lots of WorkaroundsUnmaintainable codebaseCollaboration difficulties
5 / 44
Commonpitfalls
4 / 44
Pitfalls
Cause & effect
Quick Wins
Fix your codebase!Quick wins:
Move data to Hiera
Implement Code Review
Use Puppet-lint in a git-hook
REFACTOR CONSTANTLYREFACTOR CONSTANTLY
6 / 44
Pitfalls
Cause & effectPitfalls
Lots of WorkaroundsUnmaintainable codebaseCollaboration difficulties
5 / 44
A Manageable DesignSeptember 2014 edition
7 / 44
Pitfalls
Cause & effect
Quick Wins
Fix your codebase!Quick wins:
Move data to Hiera
Implement Code Review
Use Puppet-lint in a git-hook
REFACTOR CONSTANTLYREFACTOR CONSTANTLY
6 / 44
RequirementsWhadda we need
8 / 44
A Manageable DesignSeptember 2014 edition
7 / 44
Our environment should be:Easy to UseUse
Easy to ComprehendComprehend
Easy to UpdateUpdate
and... SafeSafe
9 / 44
RequirementsWhadda we need
8 / 44
This stuff isn'texactly easy
10 / 44
Our environment should be:Easy to UseUse
Easy to ComprehendComprehend
Easy to UpdateUpdate
and... SafeSafe
9 / 44
But we cán make it safesafe andmanageablemanageable
11 / 44
This stuff isn'texactly easy
10 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
SafeUse environments to test everything
Create a huge testing environment
Use Git to promote your code
12 / 44
But we cán make it safesafe andmanageablemanageable
11 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
Manageable
ManageableKeep a consistent module structure
Using roles for abstraction
Facilitate collaboration
13 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
SafeUse environments to test everything
Create a huge testing environment
Use Git to promote your code
12 / 44
DomainsServer Roles
All things data
Deployment & Workflow
14 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
Manageable
ManageableKeep a consistent module structure
Using roles for abstraction
Facilitate collaboration
13 / 44
OverviewSoftware Components
15 / 44
DomainsServer Roles
All things data
Deployment & Workflow
14 / 44
Software ComponentsPuppet Enterprise or The Foreman
Hiera and hiera-eyaml (Hierarchical Data lookup)
Gerrit (Code review system)
Git (what else?)
Git Flow, adapted version for Gerrit
R10K (Environment deployment tool)16 / 44
OverviewSoftware Components
15 / 44
Domain #1:
Server Roles
17 / 44
Software ComponentsPuppet Enterprise or The Foreman
Hiera and hiera-eyaml (Hierarchical Data lookup)
Gerrit (Code review system)
Git (what else?)
Git Flow, adapted version for Gerrit
R10K (Environment deployment tool)16 / 44
A layer of abstraction
18 / 44
Domain #1:
Server Roles
17 / 44
How to do it?Create roles moduleroot@puppet# puppet module generate gerapeldoorn-role
Create a base-role to cover generic settings# modules/role/manifests/base.pp:class role::base { include users include ssh include motd ...
19 / 44
A layer of abstraction
18 / 44
How to do it? -Cont'd-Put all required resources in the classes# modules/role/manifests/app.pp:class role::app { include apache include tomcat apache::virtualhost { 'default': ...
Include role in node definition# site.pp:node 'app01.autiplan.com' { include role::base include role::app}
20 / 44
How to do it?Create roles moduleroot@puppet# puppet module generate gerapeldoorn-role
Create a base-role to cover generic settings# modules/role/manifests/base.pp:class role::base { include users include ssh include motd ...
19 / 44
Domain #2:
All things Data
21 / 44
How to do it? -Cont'd-Put all required resources in the classes# modules/role/manifests/app.pp:class role::app { include apache include tomcat apache::virtualhost { 'default': ...
Include role in node definition# site.pp:node 'app01.autiplan.com' { include role::base include role::app}
20 / 44
HieraHierarchical data lookup tool
22 / 44
Domain #2:
All things Data
21 / 44
Configured Hierarchy:#/etc/puppet/hiera.yaml::hierarchy: - "%{::clientcert}" - "%{::environment}" - common
Node app01.autiplan.com:
environment: testing
Hieradata# hiera/app01.autiplan.com.yaml---examplekey: value for \ app01.autiplan.com
# hiera/testing.yaml---examplekey: value for nodes in \ testing environment
# hiera/common.yaml---examplekey: value for all nodes
It's all about Hierarchy
What will be in $test?$test = hiera('examplekey')
23 / 44
HieraHierarchical data lookup tool
22 / 44
Types of HieradataRegular values# hiera/app01.autiplan.com.yaml---examplekey: value
24 / 44
Configured Hierarchy:#/etc/puppet/hiera.yaml::hierarchy: - "%{::clientcert}" - "%{::environment}" - common
Node app01.autiplan.com:
environment: testing
Hieradata# hiera/app01.autiplan.com.yaml---examplekey: value for \ app01.autiplan.com
# hiera/testing.yaml---examplekey: value for nodes in \ testing environment
# hiera/common.yaml---examplekey: value for all nodes
It's all about Hierarchy
What will be in $test?$test = hiera('examplekey')
23 / 44
Types of HieradataArrays# hiera/app01.autiplan.com.yaml---array: [ item1, item2, item3 ]
otherarray: - item1 - item2 - item3
Note: Never use tabs in Hiera files!
25 / 44
Types of HieradataRegular values# hiera/app01.autiplan.com.yaml---examplekey: value
24 / 44
Types of HieradataHashes# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value
26 / 44
Types of HieradataArrays# hiera/app01.autiplan.com.yaml---array: [ item1, item2, item3 ]
otherarray: - item1 - item2 - item3
Note: Never use tabs in Hiera files!
25 / 44
Types of HieradataCombinations# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value key3: - arrayvalue1 - arrayvalue2 key4: subhashkey1: value subhashkey2: value
27 / 44
Types of HieradataHashes# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value
26 / 44
Hiera-related functions...and what to use them for
28 / 44
Types of HieradataCombinations# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value key3: - arrayvalue1 - arrayvalue2 key4: subhashkey1: value subhashkey2: value
27 / 44
What does it do?Retrieves the first-found value in thehierarchy. (top-down)
What to use it for?Basic variable-lookup.Very easy to create exceptions!
How to use it?
$smarthost = hiera('smarthost')
Example Hieradata# hiera/mail.autiplan.com.yaml---smarthost: smtp.myprovider.nl
# hiera/testing.yaml---smarthost: testsmtp.autiplan.com
# hiera/common.yaml---smarthost: mail.autiplan.com
hiera('key' [, default_value])
29 / 44
Hiera-related functions...and what to use them for
28 / 44
What does it do?Retrieves an array or hash valuein the hierarchy, concatinates allfound results
What to use it for?Combining data from allhierarchy levels.
How to use it?
$users = hiera_array('users')
Example Hieradata# hiera/app01.autiplan.com.yaml---users: [ 'user1', 'user2' ]
# hiera/testing.yaml---users: [ 'testuser' ]
# hiera/common.yaml---users: [ 'user3', 'user4' ]
hiera_array('key' [, default_value]) (and hiera_hash)
30 / 44
What does it do?Retrieves the first-found value in thehierarchy. (top-down)
What to use it for?Basic variable-lookup.Very easy to create exceptions!
How to use it?
$smarthost = hiera('smarthost')
Example Hieradata# hiera/mail.autiplan.com.yaml---smarthost: smtp.myprovider.nl
# hiera/testing.yaml---smarthost: testsmtp.autiplan.com
# hiera/common.yaml---smarthost: mail.autiplan.com
hiera('key' [, default_value])
29 / 44
What does it do?Includes all classes listed in thearray that is loaded from Hiera.Takes elements from ALLhierarchy levels.
What to use it for?Lightweight ENC.Put all classes / roles in Hiera.
How to use it?
node default { hiera_include('roles')}
Example Hieradata# hiera/web01.autiplan.com.yaml---roles: - role::web
# hiera/common.yaml---roles: - role::base
hiera_include('classes')
31 / 44
What does it do?Retrieves an array or hash valuein the hierarchy, concatinates allfound results
What to use it for?Combining data from allhierarchy levels.
How to use it?
$users = hiera_array('users')
Example Hieradata# hiera/app01.autiplan.com.yaml---users: [ 'user1', 'user2' ]
# hiera/testing.yaml---users: [ 'testuser' ]
# hiera/common.yaml---users: [ 'user3', 'user4' ]
hiera_array('key' [, default_value]) (and hiera_hash)
30 / 44
What does it do?Generates resources from aHASH.
What to use it for?Generate any resource based ondata from Hiera.Can also be used withhiera_hash to create resourcesfrom all levels!
How to use it?
create_resources ('apache::vhost', hiera('vhosts', {}))
Example Hieradata# hiera/web01.autiplan.com.yaml---vhosts: autiplan.com: alias: www.autiplan.com autiplan.dk: alias: www.autiplan.dk docroot: /var/www/html/autiplan.dk autiplan.nl: alias: www.autiplan.nl cdn.autiplan.com: port: 81 docroot: /var/www/html/cdn
create_resources('type', HASH [, default_values])
32 / 44
What does it do?Includes all classes listed in thearray that is loaded from Hiera.Takes elements from ALLhierarchy levels.
What to use it for?Lightweight ENC.Put all classes / roles in Hiera.
How to use it?
node default { hiera_include('roles')}
Example Hieradata# hiera/web01.autiplan.com.yaml---roles: - role::web
# hiera/common.yaml---roles: - role::base
hiera_include('classes')
31 / 44
Data bindingsAuto-loading of Hiera data for parameterized classes.
33 / 44
What does it do?Generates resources from aHASH.
What to use it for?Generate any resource based ondata from Hiera.Can also be used withhiera_hash to create resourcesfrom all levels!
How to use it?
create_resources ('apache::vhost', hiera('vhosts', {}))
Example Hieradata# hiera/web01.autiplan.com.yaml---vhosts: autiplan.com: alias: www.autiplan.com autiplan.dk: alias: www.autiplan.dk docroot: /var/www/html/autiplan.dk autiplan.nl: alias: www.autiplan.nl cdn.autiplan.com: port: 81 docroot: /var/www/html/cdn
create_resources('type', HASH [, default_values])
32 / 44
What does it do?Automatically loads classparameters from Hiera.
What to use it for?Specify all class parameters inHiera.Use all hierarchical benefits forclass parameters.Simplify the use ofparameterized classes.
How to use it?
include mysql::server
Example Hieradata# hiera/web01.autiplan.com.yaml---mysql::server::root_password: m0ars3cr3t
# hiera/common.yaml---mysql::server::root_password: t0ps3cr3tmysql::server::package_name: mysql-servermysql::server::restart: true
Data bindings
34 / 44
Data bindingsAuto-loading of Hiera data for parameterized classes.
33 / 44
Putting it all togetherAnything node-specific should be in Hiera!
35 / 44
What does it do?Automatically loads classparameters from Hiera.
What to use it for?Specify all class parameters inHiera.Use all hierarchical benefits forclass parameters.Simplify the use ofparameterized classes.
How to use it?
include mysql::server
Example Hieradata# hiera/web01.autiplan.com.yaml---mysql::server::root_password: m0ars3cr3t
# hiera/common.yaml---mysql::server::root_password: t0ps3cr3tmysql::server::package_name: mysql-servermysql::server::restart: true
Data bindings
34 / 44
A Puppet Run: What calls what?
36 / 44
Putting it all togetherAnything node-specific should be in Hiera!
35 / 44
Domain #3:
Deployment & Workflow
37 / 44
A Puppet Run: What calls what?
36 / 44
EnvironmentsKeeping the environmentalists happy
38 / 44
Domain #3:
Deployment & Workflow
37 / 44
EnvironmentsWhat is an environment?
Seperate modulepaths/site.pp.Common environments: development, testing, production.Nodes request a specific environment.
Why?Essential to prevent mistakes.NEVER edit code in production!The workflow helps us to 'promote' our code to production.
39 / 44
EnvironmentsKeeping the environmentalists happy
38 / 44
Demo!
40 / 44
EnvironmentsWhat is an environment?
Seperate modulepaths/site.pp.Common environments: development, testing, production.Nodes request a specific environment.
Why?Essential to prevent mistakes.NEVER edit code in production!The workflow helps us to 'promote' our code to production.
39 / 44
R10k overview
41 / 44
Demo!
40 / 44
Final remarksKeep public modules as-is, wherever possible
Create wrapper classes in company-module.Create fork if needed, submit pull request for fixes.
Add forked module (gitrepo) to Puppetfile.
Think aheadAlways try to anticipate future applications.If it feels overly complicated, yer doin it wrong.Refactor!
42 / 44
R10k overview
41 / 44
Questions?
43 / 44
Final remarksKeep public modules as-is, wherever possible
Create wrapper classes in company-module.Create fork if needed, submit pull request for fixes.
Add forked module (gitrepo) to Puppetfile.
Think aheadAlways try to anticipate future applications.If it feels overly complicated, yer doin it wrong.Refactor!
42 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Thank you!A howto of setting up this environment (and the workflow!) is available on my
blog: http://puppetspecialist.nl/mpi
44 / 44
Questions?
43 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Thank you!A howto of setting up this environment (and the workflow!) is available on my
blog: http://puppetspecialist.nl/mpi
44 / 44