Growing Pains: Avoiding Module Hell

transcript

Growing Pains: Avoiding Module

Hell Chad Metcalf

@metcalfc

Saturday, August 24, 13

As do many stories, it all begins with ...

$ sudo

sudo::conf { 'web': source => 'puppet:///files/etc/sudoers.d/web', } sudo::conf { 'admins': priority => 10, content => "%admins ALL=(ALL) NOPASSWD: ALL\n", } sudo::conf { 'joe': priority => 60, source => 'puppet:///files/etc/sudoers.d/users/joed', }

sudo::rule { "extra_rule": ensure => present, who => 'bob', commands => "/usr/sbin/systemctl", nopass => false, comment => "what ever you like", }

Infrastructure as Code

Module as an API

Source Code Management

Verification and Validation

Release Management

semver.org

MAJOR.MINOR.PATCH

MAJOR version when you make incompatible API changes

MINOR version when you add functionality in a backwards-compatible manner

PATCH version when you make backwards-compatible bug fixes

Its OK to go:

Easy to learnEasy to useHard to misuseEasy to

Gather requirementsUse casesDon’t over specifyPrivate as possible

Nifty::Firewall

Nifty::Monitoring

Nifty::Supervisor

iptables

Icinga

Firewall::Nifty

Monitoring::Nifty

Supervisor::Nifty

Nifty::Firewall

Nifty::Monitoring

Nifty::Supervisor

Supervisord

Firewall::Nifty

Monitoring::Nifty

Supervisor::Nifty

Don’t Specify ALL THE THINGS!!!

define apache::vhost ( $server_admin $server_name = '', $docroot = '', $docroot_create = false, $docroot_owner = 'root', $docroot_group = 'root', $port = '80', $ssl = false, $template = 'apache/virtualhost/vhost.conf.erb', $priority = '50', $serveraliases = '', $env_variables = '', $passenger = false, $passenger_high_performance = true, $passenger_max_pool_size = 12, $passenger_pool_idle_time = 1200, $passenger_max_requests = 0, $passenger_stat_throttle_rate = 30, $passenger_rack_auto_detect = true, $passenger_rails_auto_detect = false, $passenger_rails_env = '',23 $passenger_rails_base_uri = '',24 $passenger_rack_env = '',25 $passenger_rack_base_uri = '',26 $enable = true,27 $directory = '',28 $directory_options = '',29 $directory_allow_override = 'None'30 )

<?xml version="1.0" encoding="UTF-8"?> 2 3  4 <configuration> 5 <property> 6 <name>mapred.job.tracker</name> 7 <value>xwing07.ul.wibidata.net:8021</value> 8 </property> 9 <property>10 <name>mapreduce.job.counters.max</name>11 <value>120</value>12 </property>13 <property>14 <name>mapred.output.compress</name>15 <value>false</value>16 </property>17 <property>18 <name>mapred.output.compression.type</name>19 <value>BLOCK</value>20 </property>21 <property>22 <name>mapred.output.compression.codec</name>23 <value>org.apache.hadoop.io.compress.DefaultCodec</value>24 </property> <property> <name>mapred.map.output.compression.codec</name> <value>org.apache.hadoop.io.compress.SnappyCodec</value> </property> <property> <name>mapred.compress.map.output</name> <value>true</value> </property> <property> <name>io.sort.factor</name> <value>64</value> </property> <property> <name>io.sort.record.percent</name> <value>0.05</value> </property> <property> <name>io.sort.spill.percent</name> <value>0.8</value> </property> <property> <name>mapred.reduce.parallel.copies</name>47 <value>10</value>48 </property>49 <property>50 <name>mapred.submit.replication</name>51 <value>2</value>52 </property>53 <property>54 <name>mapred.reduce.tasks</name>55 <value>24</value>56 </property>57 <property>58 <name>io.sort.mb</name>59 <value>233</value>60 </property>61 <property>62 <name>mapred.child.java.opts</name>63 <value> -Xmx978972870</value>64 </property>65 <property>66 <name>mapred.job.reuse.jvm.num.tasks</name>67 <value>1</value>68 </property>69 <property>70 <name>mapred.map.tasks.speculative.execution</name>71 <value>false</value>72 </property>73 <property>74 <name>mapred.reduce.tasks.speculative.execution</name>75 <value>false</value>76 </property>77 <property>78 <name>mapred.reduce.slowstart.completed.maps</name>79 <value>0.8</value>80 </property>81 </configuration>

{ "foobar-cluster": { "mapred-options": [ { "name": "mapred.map.output.compression.codec", "value": "org.apache.hadoop.io.compress.SnappyCodec" }, { "name": "mapred.compress.map.output", "value": "true" }, { "name": "io.sort.factor", "value": "64" }, { "name": "io.sort.record.percent", "value": "0.05" }, { "name": "mapred.map.tasks.speculative.execution", { "name": "mapred.reduce.tasks.speculative.execution",54 "value": "false"55 },56 {57 "name": "mapred.reduce.slowstart.completed.maps",58 "value": "0.8"59 }60 ]61 }62 }

Growing Pains: Avoiding Module Hell

Technology