Date post: | 22-Jun-2015 |
Category: |
Technology |
Upload: | puppet-labs |
View: | 3,950 times |
Download: | 0 times |
2014
Presented by
How Puppet Enables the Use of Lightweight Virtualized ContainersJeff McCune
Software Developer | Puppet Labs
@0xEFF
Presented by
What we’ll coverCompare and contrasts a Jenkins+LDAP deployed onto virtual machines and onto lightweight containers.
!
Jenkins and LDAP Puppet Forge Modules Migrating from a shared VM to containers Common Problems with Service Management Summary of Lessons Learned
Presented by
Starting Pointhttps://github.com/jeffmccune/puppetconf2014
Traditional VM
Puppet deploys Jenkins with LDAP security
http://sharedvm:18080
Presented by
Shared VMVagrant VM
OpenLDAPJenkins
Puppet
Puppet configures Jenkins and OpenLDAP
Vagrant Puppet provisioner
Presented by
Migrating to ContainersVagrant VM
OpenLDAPJenkins
Puppet at runtime
Docker Host
!
!
OpenLDAP Container
!
!
Jenkins Container
Puppet at build time
Puppet at build time
Presented by
Puppet Forge module re-useIt is faster to deploy Jenkins with LDAP using forge modules:
rtyler/jenkins module 90% code re-use: 831 lines from the Forge, 88 lines added
camptocamp/openldap module85% code re-use: 584 lines from the Forge, 108 lines added
Presented by
Site Specific CustomizationsInitialize LDAP for dc=jeffmccune,dc=net
Configure LDAP admin account and password
Load initial schema and base OU’s into LDAP
Configure Jenkins to use LDAP security
Presented by
Moving from a VM to a ContainerChallenges in Puppet:
Service resources fail and must be overridden
Downstream resources require the service resource
LDAP initial schema load requires Service[slapd]
Presented by
Why do service resources fail? Two different models of containers:
Those with and without a service management framework.
In either case there is no service management framework running during the image build phase.
Presented by
ContainersTwo Models of Containers:
OS Virtualization Model - Heavy(Solaris Zones, FreeBSD Jails, IBM WPAR’s)
Microservice Model - Light (Docker, CoreOS, Kubernetes)
Presented by
False Dichotomy over “Light”Weight is a range, not a boolean.
Single-process image
Multi-process image
Full-os image
Runtime manageability decreases as processes decrease
Presented by
Single-process MicroservicesIn general:
Executes the application process directly, no init system
startup time comparable to normal process startup
Puppet does not operate at runtime
Difficult to manage with volumes and sidekick processes
Presented by
Multi-process MicroservicesIn general:
Start an init system (SysV init, systemd, upstart)
The service management framework manages services
Runs ~5 or more processes per container
Puppet works at runtime without modification
Presented by
Full OS Virtualization ContainersRun everything a traditional OS would run
Dozens of running processes in the container
Long startup time (> 30 seconds)
Easier to manage at runtime
Better for some situations, e.g. mimicking a full running OS
Presented by
Container ImagesMicroservice containers have a distinct build phase
The result of the build step is an image, ideally immutable
Instances execute from a built base image
The Dockerfile is an example of a build script
Presented by
DockerfileFROM centos:centos7
ADD puppet.tar.gz /
RUN puppet apply -v --modulepath=/puppet/modules \ /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/init"]
Presented by
DockerfileFROM centos:centos7
ADD puppet.tar.gz /
RUN puppet apply -v --modulepath=/puppet/modules \ /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/init"]
Presented by
DockerfileFROM centos:centos7
ADD puppet.tar.gz /
RUN puppet apply -v --modulepath=/puppet/modules \ /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/init"]
Presented by
DockerfileFROM centos:centos7
ADD puppet.tar.gz /
RUN puppet apply -v --modulepath=/puppet/modules \ /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/init"]
Presented by
docker build$ docker build --tag base_image . Sending build context to Docker daemon 4.718 MB Step 0 : FROM centos:centos7 ---> 3236320e3562 Step 1 : ADD puppet.tar.gz / ---> bc9aa11de092 Step 2 : RUN puppet apply … ---> Running in efdda7633bfc
Presented by
docker build$ docker build --tag base_image . Sending build context to Docker daemon 4.718 MBStep 0 : FROM centos:centos7 ---> 3236320e3562 Step 1 : ADD puppet.tar.gz / ---> bc9aa11de092 Step 2 : RUN puppet apply … ---> Running in efdda7633bfc
Presented by
docker build$ docker build --tag base_image . Sending build context to Docker daemon 4.718 MB Step 0 : FROM centos:centos7 ---> 3236320e3562 Step 1 : ADD puppet.tar.gz / ---> bc9aa11de092 Step 2 : RUN puppet apply … ---> Running in efdda7633bfc
Presented by
docker build$ docker build --tag base_image . Sending build context to Docker daemon 4.718 MB Step 0 : FROM centos:centos7 ---> 3236320e3562 Step 1 : ADD puppet.tar.gz / ---> bc9aa11de092 Step 2 : RUN puppet apply … ---> Running in efdda7633bfc
Presented by
Docker Build with Puppet Try #1Error: /Service[slapd]: Could not evaluate: Could not find init script for 'slapd'
Warning: Openldap_database[dc=jeffmccune,dc=net]: Skipping because of failed dependencies
Warning: Exec[inetorgperson schema]: Skipping because of failed dependencies
And so on for all dependent resources…
Presented by
systemd is not installedThe error, Could not find init script for ‘slapd’ is caused by a fake systemd in the base centos image
$ docker run -i -t centos rpm -qa | grep systemd fakesystemd-1-15.el7.centos.noarchsystemd-libs-208-11.el7_0.2.x86_64
Presented by
Fix #1 Install the real systemdReplace fakesystemd with the real deal
microservice => virtualized os model
Start a minimum number of services with systemd
New base image: centos7vps with systemd
Presented by
Fix #1 DockerfileFROM centos:centos7
RUN yum -y swap \ -- remove fakesystemd \ -- install systemd systemd-libs
# RUN rm unit files in /{lib,etc}/systemd/system
CMD ["/usr/sbin/init"]
Presented by
Build new base image$ docker build -t centos7vps .
This base image has the real systemd installed.
Presented by
New DockerfileFROM centos7vps # <= was centos:centos7
ADD puppet.tar.gz /
RUN puppet apply -v --modulepath=/puppet/modules \ /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/init"]
Presented by
Working for docker run…Puppet works inside a running docker container
systemd is running
Presented by
Working for docker run…Start the systemd container
$ CID=$(docker run --privileged -d centos7vps)
Enter the container
$ sudo /usr/local/bin/docker-enter $CID
Presented by
Working for docker run…Run Puppet:
bash-4.2# puppet apply /puppet/manifests/ldap.pp Compiled catalog for ldap in 1.05 seconds Service[slapd]/ensure: 'stopped' => 'running' Finished catalog run in 30.15 seconds
Presented by
Not working for docker buildsystemd is present, but not running during the build phase
$ docker build -t ldap:vps . …Error: Could not start Service[slapd]: Execution of '/usr/bin/systemctl start slapd'returned 1: Failed to get D-Bus connection: No connection to service manager.
Presented by
Moving aheadWe could stop now and simply run puppet every time the container starts prior to the application starting.
This would increase service start time.
Application images become mutable, managed by Puppet.
Doesn’t fit the microservice model very well.
Presented by
Goal: Puppet configured images Puppet configures immutable pre-configured base application images
Immutable images provide known good state
Closer to the microservice model
Compatible with the os virtualization model
Presented by
Docker build with Puppet Try #2Override the Service[slapd] to avoid systemd at build time
class ldap_override inherits openldap::server::service { Service[slapd] { ensure => running, enable => undef, start => '/usr/sbin/slapd -u ldap -h "ldapi:/// ldap:///" } }
Presented by
Success!We’re able to build an immutable application image fully configured by Puppet.
Presented by
LDAP running in systemd/usr/bin/docker -d --selinux-enabled \_ /usr/sbin/init \_ /usr/lib/systemd/systemd-journald \_ /bin/dbus-daemon --system \_ /usr/sbin/slapd -u ldap -h ldapi:/// ldap:///
Presented by
Moving further aheadWe could stop here, but there are a number of trade-offs
systemd requires the container to run in privileged mode
More processes: systemd, journald, dbus, application
Slower startup / tear-down time (~5 seconds)
Still faster than a VM (~30+ seconds)
Presented by
Final DockerfileFROM centos7vps
ADD puppet.tar.gz /
RUN puppet apply -v /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/slapd", "-u", "ldap", \ "-h", "ldapi:/// ldap:///", "-d", "0"]
Presented by
Final DockerfileFROM centos7vps
ADD puppet.tar.gz /
RUN puppet apply -v /puppet/manifests/ldap.pp
EXPOSE 389
CMD ["/usr/sbin/slapd", "-u", "ldap", \ "-h", "ldapi:/// ldap:///", "-d", "0"]
Presented by
docker build ldap:microserviceBuild is required for deployment
$ docker build -t ldap:microservice .
Presented by
Single-process Microservice Only one process running in the container:
/usr/bin/docker -d --selinux-enabled \_ /usr/sbin/slapd -u ldap -h ldapi:/// ldap:/// -d 0
Start / Stop time: ~1 second
Presented by
Puppetized MicroserviceThe application image is immutable
Changes to the application require a new image build
Centralized reporting via Puppet
version control via Puppet code
Re-usable modules from the Puppet forge
Presented by
Putting it togetherLink the jenkins microservice with the ldap microservice
$ docker run -d --name ldap ldap:microservice $ docker run -d --name jenkins \ --link ldap:ldap \ --publish 18080:8080 \ jenkins:microservice
Presented by
ResultsThe application (jenkins) is now isolated
With isolated dependencies included (ldap)
The applications are immutable
Multiple copies can easily be deployed
Difficult to change running instances
Presented by
Making a change to LDAPNeed to modify the LDAP indexes as an optimization
1: Update Puppet Configuration
2: Rebuild LDAP image
3: Re-deploy LDAP
4: Re-deploy Jenkins (The link to LDAP is static)
Presented by
Advantages and DisadvantagesImmutable known good state
Deployment is highly repeatable and consistent
App and Dependency are tightly coupled
Ambassador Pattern decouples services (with tedium)
!
Presented by
SummaryForge modules are re-usable in Docker containers
Service resources pose a challenge
Override the service resource to build immutable images
Multi-process images with systemd are an option
Presented by
Thank you!Code: github.com/jeffmccune/puppetconf2014
David Lutterkort’s Dockercon talk: http://links.puppetlabs.com/lutter-docker
!