ConfigManagement mit Pallet - XP Days · 2017-05-19 · Stuttgart & Meetup IT-Architektur Süd...

Michael Jergerwww.DomainDrivenArchitecture.org

TestDrivenDevOps_0_521/11/16 / Seite 1

Test-Driven DevOps

Michael Jerger



Agenda

Vorstellung

Testing für Developer

Herausforderung Operations

Ansatzpunkte für agile DevOps

Testen in dda-pallet



Wer spricht? - meissa GmbH

Fokus: DevOps, Maintainer von dda-pallet

Sitz Reutlingen, SW-Entwicklung Java & Uni Tübingen

Arbeiten dezentral, d.h. 80% Remote-Arbeit

Zusammenarbeit auf Augenhöhe

https://www.meissa-gmbh.de



DomainDrivenArchitecture?

Kollaborative Architektur, OpenSource

Zusammenarbeit aus AG-IT-Architektur Stuttgart & Meetup IT-Architektur Süd

Inspiriert durch DomainDrivenDesign von E.Evans

https://www.DomainDrivenArchitecture.org



Agenda

Vorstellung







Testarten für Dev



Agenda

Vorstellung







naiv?

Naiv landen wir komplett in Infrastruktur Integration!



Ops = viel State

Installierte Betriebssystem, Software

System-Konfiguration

Update-Zustand

Installierte Applikationen und deren Zustand

Logfiles und deren Nutzung



State verbessern

State isolieren

Container

CM State managen

Dockerhost

ZabbixServer

Maria DB



Ops = Heterogene Technik

Unix / Windows/ *sh / Powershell

Perl / PHP / C / Java / Python / Ruby / GO / Lisp / JavaScript

Make / ant / mvn / gradle / lein / grunt

Config von HW Firewall / Router / VPN

Config von Applikationen



Heterogenität verbessern

Trennung: Konventions- vs. System-Module

httpd { :limits {:serverlimit 150 :maxclients 150} :jkconfiguration {:jkStripSession "On" :jkWatchdogInterval 120} :vhosts {:default {:domainname "localhost.localdomain" :listeningport "443" :serveradminemail "admin@localdomain" :maintainancepagecontent ["<h1>Maintainance Mode</h1>"] :modjk {:port "8009" :host "127.0.0.1" :worker "mod_jk_www" :maintaintimoutsec 90 :socketconnecttimeoutms 62000} }}

System:



Heterogenität verbessern

Trennung v. Konvention & System

target { :name „monitoring“ :app :zabbix :stage :prod}

httpd { :limits {:serverlimit 150 :maxclients 150} :jkconfiguration {:jkStripSession "On" :jkWatchdogInterval 120} :vhosts {:default {:domainname "localhost.localdomain" :listeningport "443" :serveradminemail "admin@localdomain" :maintainancepagecontent ["<h1>Maintainance Mode</h1>"] :modjk {:port "8009" :host "127.0.0.1" :worker "mod_jk_www" :maintaintimoutsec 90 :socketconnecttimeoutms 62000} }}

Konvention: System:



Ops = Infrastructure

Netzwerk, Firewall, Routing

Integration in Ops Services

Repositories: Linux-Packages, Software-Artefakte, Web-Downloads

Security: Krypto-Keys & Credentials



Infrastructure verbessern

Immer wieder dasselbe

automatisiert & getestettun ...



Agenda

Vorstellung







CM - Unittest



Fkt. Integrationstest



ServerSpec

describe port(80) do it { should be_listening }end



Integrationstest

Local - Whitebox



Integrationstest

Remote - Whitebox



Integrationstest

Remote - Blackbox



Agenda

Vorstellung







dda-pallet für puppet-KennerRuby

Go

Rake

Puppet

Kitchen

Vagrant

ServerSpec

Packer

terraform

Clojure

Lein

dda-pallet



Konzepte: Kommunikation



Bash AbstraktionNennt sich in Pallet: Stevedore

Schleifen

Funktionen

Variablen

(script (doseq [x ["a" "b" "c"]] (println @x)))

for x in a b c; do echo ${x}done

(script (defn foo [x y] ("bar" x)))

script function foo() { x=$1 y=$2 bar x}

(let [tmp "/tmp"] (script ("ls" ~tmp)))

ls /tmp



… it's Clojure, stupid ...Compiliert auf die jvm oder JavaScript

Funktionen, Typen, Interfaces, erweiterte Vererbung, ...

Validierung (Typisierung mit schema)

Module, Artefakte, Build (lein vgl. mvn)

Test First, Unit-Tests

Debugging, Codevervollständigung, REPL ...



Recap: Test Kontext



System-Adapter: File Inhalt

=> (vhost/vhostconfdefaultredirecttohttpsonly :domainname "meissagmbh.de" :serveradminemail "admin@meissagmbh.de")["<VirtualHost *:80>" " ServerName meissagmbh.de" " ServerAdmin admin@meissagmbh.de" " " " ErrorLog \"/var/log/apache2/error.log\"" " LogLevel warn" " CustomLog \"/var/log/apache2/access.log\" combined" " " " RewriteEngine on" " RewriteCond %{HTTPS} !on" " RewriteRule ^/(.*)$ https://%{SERVER_NAME}/$1 [R=301,L]" " " "</VirtualHost>"]

REPL

https://github.com/DomainDrivenArchitecture/dda-httpd-crate/blob/master/test/org/domaindrivenarchitecture/pallet/crate/httpd/vhost_multi_test.clj

https://github.com/DomainDrivenArchitecture/dda-httpd-crate/blob/master/test/org/domaindrivenarchitecture/pallet/crate/httpd/vhost_multi_test.clj

System Adapter: Plan

https://github.com/DomainDrivenArchitecture/dda-init-crate/blob/credentials/integration/org/domaindrivenarchitecture/pallet/crate/instantiate_aws_init.clj



Konvention

(def archivaconfig {:fqdn "artifacts.meissagmbh.de" :smtpmailhost "mailserver" :smtpmailuser "mailaccount" :smtpmailpassword "mailpwd"})

=> (sut/archivahttpdconfig archivaconfig){:limits {:serverlimit 150, :maxclients 150}, :jkconfiguration {:jkStripSession "On", :jkWatchdogInterval 120}, :vhosts {:default {:domainname "artifacts.meissagmbh.de", :listeningport "443", :serveradminemail "[email protected]gmbh.de", :maintainancepagecontent ["<h1>Webserver Maintainance Mode</h1>"], :modjk {:port "8009", :host "127.0.0.1", :worker "mod_jk_www", :maintaintimoutsec 90, :socketconnecttimeoutms 62000}, :certletsencrypt {:letsencryptmail "[email protected]gmbh.de"}}}}

REPL



Konvention: Unit-Test failed

REPL

(deftest testconvention [] (testing "httpd convention test" (is (= {:limits ..., :vhosts {:default {:domainname "artyfacts.meissagmbh.de", ...}, :certletsencrypt {:letsencryptmail "[email protected]gmbh.de"}}}} (sut/archivahttpdconfig archivaconfig) )) ))

=> (runtests)

Testing de.meissagmbh.pallet.crate.archiva.apptest

FAIL in (testconvention) (web_test.clj:19)httpd convention testExpected: ... Actual: ... diff: {:vhosts {:default {:domainname "artyfacts.meissagmbh.de"}}} + {:vhosts {:default {:domainname "artifacts.meissagmbh.de"}}}

Ran 1 tests containing 1 assertions.1 failures, 0 errors.{:test 1, :pass 0, :fail 1, :error 0, :type :summary}

(runtests)



Konvention: Unit Test fixed

REPL

(deftest testconvention [] (testing "httpd convention test" (is (= {:limits ..., :vhosts {:default {:domainname "artifacts.meissagmbh.de", ...}, :certletsencrypt {:letsencryptmail "[email protected]gmbh.de"}}}} (sut/archivahttpdconfig archivaconfig) )) ))

=> (runtests)

Testing de.meissagmbh.pallet.crate.archiva.apptest

Ran 1 tests containing 1 assertions.0 failures, 0 errors.{:test 1, :pass 1, :fail 0, :error 0, :type :summary}

(runtests)



Image: Integrationstest



Image: Resource on target

(defn define-resources-netstat [] (define-resource-from-script

res-id-open-ports "netstat -tulpen"))

Proto RecvQ SendQ Local Address Foreign Address State User Inode PID/Program nametcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 0 9807 1001/sshd tcp6 0 0 :::80 :::* LISTEN 0 44161 4135/apache2 tcp6 0 0 :::4369 :::* LISTEN 108 33687 27416/epmd



Image: skip first line on CI

(rest netstat-resource)

=> (rest netstat-resource)("tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 0 9807 1001/sshd" "tcp6 0 0 :::80 :::* LISTEN 0 44161 4135/apache2" "tcp6 0 0 :::4369 :::* LISTEN 108 33687 27416/epmd ")

REPL



Image: parse on CI

(clojure.string/split "tcp6 0 0 :::80 :::* LISTEN 0 44161 4135/apache2" #"\s+|/")

REPL

=> (clojure.string/split "tcp6 0 0 :::80 :::* LISTEN 0 44161 4135/apache2" #"\s+|/")["tcp6" "0" "0" ":::80" ":::*" "LISTEN" "0" "44161" "4135" "apache2"]



Image: assign names

(map #(zipmap [:proto :recv-q :send-q :local-adress :foreign-adress :state :user :inode :pid :process-name] (parse-netstat netstat-resource)))

REPL

...{:foreign-adress ":::*", :local-adress ":::80", :recv-q "0", :inode "44161", :state "LISTEN", :process-name "apache2", :proto "tcp6", :pid "4135", :send-q "0", :user "0"}...



Image: apache listen?(and (= (:state named-netastat-line) "LISTEN") (= (:process-name named-netastat-line) "apache2") (some? (re-matches (re-pattern (str ".+:" 80)) (:local-address named-netastat-line))))

REPL

=> (and (= (:state named-netastat-line) "LISTEN") (= (:process-name named-netastat-line) "apache2") (some? (re-matches (re-pattern (str ".+:" 80)) (:local-address named-netastat-line))))true



Instance creation

Statt Continous Integration testet nun das

provisionierende System



Fazit

Respekt: Dev und Ops unterscheiden sich ...

Fail Fast: Ist auch für DevOps möglich

Es gibt effiziente Testmöglichkeiten



Kontakt

[email protected]

https://www.DomainDrivenArchitecture.org

Offenes PairProgramming b. Agile Ventures

http://www.agileventures.org/projects/dda-pallet

https://www.DomainDrivenArchitecture.org/

http://www.agileventures.org/projects/dda-pallet

Date post:	02-Jun-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

ConfigManagement mit Pallet - XP Days · 2017-05-19 · Stuttgart & Meetup IT-Architektur Süd...

Documents