Deploying with Super Cow Powers

Hosting your own APT repositorywith reprepro

Simon BouletConsultant, Deployment and Automationsimon@nostalgeek.com

DevOps MontréalFebruary 2015

1

Challenge of Modern Application

You want:- Nginx 1.7- Node.js 0.11- MongoDB 2.6- Consul

But latest Ubuntu has:- Nginx 1.4.6- Node.js 0.10.25- MongoDB 2.4.9- Consul N/A

2

/etc/apt/sources.list

3

How do you turn this:deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gendeb https://deb.nodesource.com/node-devel wheezy maindeb http://nginx.org/packages/mainline/debian/ wheezy nginxdeb http://ppa.launchpad.net/bcandrea/consul/ubuntu/ trusty main

Into this:deb http://apt.devops.quebec/ dev main

Signatures

And this:apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10apt-key adv --keyserver keyserver.ubuntu.com --recv 68576280apt-key adv --keyserver keyserver.ubuntu.com --recv 7BD9BF62apt-key adv --keyserver keyserver.ubuntu.com --recv E2FDAE02

Into this:apt-key adv --keyserver keyserver.ubuntu.com --recv ABCD1234

4

Red Pill, Blue Pill?

Who has never been through this?

5

Reprepro

- Manage your own APT repository- Allow for syncing external repos- Can do signatures checks and resign- Does NOT package .deb for you- Does NOT make your repository externally

accessible

6

Syncing External Repo

7

Reprepro: conf/updatesName: mongodbSuite: distComponents: 10gen>mainMethod: http://downloads-distro.mongodb.org/repo/debian-sysvinitVerifyRelease: 7F0CEB10

Name: nodesourceSuite: jessieComponents: mainMethod: https://deb.nodesource.com/node-develFilterFormula: Package (==nodejs)VerifyRelease: 68576280

Name: debian-20141003Suite: jessieComponents: mainMethod: http://snapshot.debian.org/archive/debian/20141003T221320Z/FilterFormula: Source (==libguestfs)VerifyRelease: 46925553

8

Reprepro: conf/distributionsCodename: devSuite: unstableArchitectures: amd64Components: mainTracking: minimalUpdate: mongodb nodesource nginx consul debian-20141003SignWith: ABCD1234

Codename: prodSuite: stableArchitectures: amd64Components: mainTracking: minimalSignWith: ABCD1234

reprepro update

9

Packaging Configurations Tricks

- Rebuild config packages simultaneously for all environments

- Bump config package version on each build- Don’t store secrets in packages- Use conf.d directories when available- Setup diversion if you really need to update

configurations files provided by other packages

10

Config Package: debian/controlSource: superapp-configSection: unknownPriority: extraMaintainer: Simon Boulet <simon@nostalgeek.com>Build-Depends: debhelper (>= 8.0.0)Standards-Version: 3.9.3Package: superapp-config-devArchitecture: allProvides: superapp-configDescription: Super App Config (dev)Package: superapp-config-prodArchitecture: allProvides: superapp-configDescription: Super App Config (prod)

11

Config Package: debian/*.install files

debian/superapp-config-dev.install:dev/etc/nginx/conf.d/ssl.conf etc/nginx/conf.ddev/etc/superapp/config.js etc/superappcommon/etc/nginx/conf.d/common.conf etc/nginx/conf.d

debian/superapp-config-prod.install:prod/etc/nginx/conf.d/ssl.conf etc/nginx/conf.dprod/etc/superapp/config.js etc/superappcommon/etc/nginx/conf.d/common.conf etc/nginx/conf.d

dh_make

12

Multiple Environments

- Use per-environment config package- Each environment to have their own suite:

deb http://apt.devops.quebec/ dev maindeb http://apt.devops.quebec/ staging maindeb http://apt.devops.quebec/ prod main

- Always add package to dev, and use copy to promote from dev to staging or prod

13

Promoting Dev > Staging > Prod

Adding to dev:reprepro includedeb dev <.deb file>

Promoting from dev to prod:reprepro copy prod dev <packages...>

14

Integrating with CM Tools

Ansible:- Add your repository (apt_repository)- Import your signing key (apt_key)- Ensure conformity (ansible --check)

15

Export your Repository

Using your favorite web server:- Make /dist and /pool folders available- Use .htaccess (or other method) for limiting

accessUsing SSH:deb ssh://repo@apt.devops.quebec/path/to/repo dev main

16

Going Large Scale

- Sync your repository to an Object Store (Amazon S3, Rackspace Cloud Files, etc.)

- Use CDN service in front of your repository (CloudFront, CloudFlare, etc.)

17

Notes on using Amazon S3

- S3 treats “+” in filename as space characters. Packages with “+” in their version numbers won’t work [1]

- No HTTP authentication on S3. See apt-transport-s3 [2] for private repo.

18

[1] https://forums.aws.amazon.com/message.jspa?messageID=208095[2] https://github.com/kyleshank/apt-transport-s3

Deploying with Super Cow Powers

- Control versions of packages in different environments (enforces deployment pipeline)

- Simplifies repo and key management by having a centralized repo

- Ease config management by packaging application configuration

19

Thank you!

Questions?

Simon Bouletsimon@nostalgeek.com

https://www.linkedin.com/in/simonboulethttps://github.com/siboulet

20