Date post: | 14-Jul-2015 |
Category: |
Technology |
Upload: | love-nyberg |
View: | 1,261 times |
Download: | 3 times |
BLOGLOVIN’
Using SaltStack to orchestrate microservices in application
containers
Love Nyberg Site reliability & Automation Engineer
#SaltConf15
BLOGLOVIN’
#SaltConf15
tens of millions of monthly unique visitors hundred of millions requests per day couple of million stories parsed per day
Hybrid DC/Cloud - Bare metal servers - Cloud VM’s
All those buzz words!
• Configuration management • Continues Integartion • Containers • Orchestration • Micro services
#SaltConf15
Evolution of container management at bloglovin?
Phase 1 Rolling deploys with static amount
Phase 2 Green/Blue deploy with dynamic pillar
#SaltConf15
Phase 1: Rolling deploys with static amount
Static amount of containers on each ship
Rolling deployes on each ship
Drawbacks no healtchecks serving diffrent version of code during deploy
#SaltConf15
Run containers
#SaltConf15
{% set name = 'node-demo' %} {% set registryname = 'jacksoncage' %} {% set tag = salt['pillar.get']('imagetag', "latest") %} {% set containerid = salt['grains.get']('id') %} {% set hostport = '808' %} {% set hostip = grains['ip_interfaces']['eth0'][0] %} {% set tag = salt[‘pillar.get']('containers:imagetag', "latest") %} {% set noofcontainers = range(10) ‰}
State file: 1. Setting variables
Run containers
#SaltConf15
{{ name }}-image: docker.pulled: - name: {{ registryname }}/{{ name }} - tag: {{ tag }} - force: True
State file: 2. Pulling docker image
Run containers
#SaltConf15
{{ name }}-stop-if-old-{{ no }}: cmd.run: - name: docker stop {{ containerid }}-{{ name }}-{{ no }} - unless: docker inspect --format '{{ .Image }}' {{ containerid }}-{{ name }}-{{ nr }} | \ grep $(docker images --no-trunc | grep "{{ registryname }}/{{ name }}" | awk '{ print $3 }') - require: - docker: {{ name }}-image
State file: 3. Stop container if newer image exists
Run containers
#SaltConf15
{{ name }}-remove-if-old-{{ no }}: cmd.run: - name: docker rm {{ containerid }}-{{ name }}-{{ no }} - unless: docker inspect --format '{{ .Image }}' {{ containerid }}-{{ name }}-{{ nr }} | \ grep $(docker images --no-trunc | grep "{{ registryname }}/{{ name }}" | awk '{ print $3 }') - require: - cmd: {{ name }}-stop-if-old-{{ no }}
State file: 4. Removing container if newer image exists
Run containers
#SaltConf15
{{ name }}-container-{{ no }}: docker.installed: - name: {{ containerid }}-{{ name }}-{{ no }} - hostname: {{ containerid }}-{{ name }}-{{ no }} - image: {{ registryname }}/{{ name }}:{{ tag }} - ports: - "8080/tcp" - environment: - EXECUTER: "forever -f start" - APP: "index.js" - require_in: {{ name }}-{{ no }} - require: - docker: {{ name }}-image
State file: 5. Installing container
Run containers
#SaltConf15
{{ name }}-{{ no }}: docker.running: - container: {{ containerid }}-{{ name }}-{{ no }} - port_bindings: "8080/tcp": HostIp: "{{ hostip }}" HostPort: "{{ hostport }}{{ no }}"
State file: 6. Start container
Run containers
#SaltConf15
salt minion1 state.sls containers.applications.node-demo 'pillar={ containers: { imagetag: 1.0.0 } }'
Run: command line
Run containers
#SaltConf15
curl -H 'X-Salt-Deploy-Key: QmpaDa8T3UdFVBUHWo@T' \ -H 'Content-Type: application/json' \ -d '{"application":"api","container":"api-phase1","node":"ship01","tag":"1.0","amount":"2"}' \ http://127.0.0.1:8000/hook/api/deploy/phase2/success
salt-call event.fire_master '{"salt-node": "ship01", "application": "api", “container": \ "api-phase1", "tag": "1.0"}' deploy/containers
Run: via api/reactor
Phase 2: Green/Blue deploy with dynamic pillar
• Deploy a new containers before killing the old once
• Use healtchecks
• Use dynamic updated pillar data with etcd
#SaltConf15
Run containers
#SaltConf15
Reactor sls
1. Sync states 2. Update etcd with tags, amount etc 3. Pull docker image 4. Deploy unactive containers to cluster 5. Test unactive containers and add to etcd 6. Update haproxy using etcd 7. Close down unactive containers
Demo
#SaltConf15
Phase 1: https://github.com/jacksoncage/docker-node-demo
Phase 2: https://github.com/jacksoncage/saltconf-container-demo