Puppet CA: Certificates explainedThomas Gelf - PuppetCamp Düsseldorf 2014

Thomas Gelf, nice to meet you!

joined NETWAYS in 2010

formerly more than ten years of...

web (application) development

routing/switching: bank/ISP backbones

ISP: Mail, Hosting, SIP-Carrier, IPv6...

Origins

nationality: Italian

mother tongue: German

kind of.

SOUTH TYROLEAN!!!

Me and Puppet

first Puppet steps with 0.24

talks, articles, blog posts

trainer, consultant

over-certified

Me @ PuppetConf 2014

Had a great time, the conference was awesome!

PuppetConf 2015 will be in Portland - see you there :)

NETWAYS

Netways and Puppet

German Puppet Labs Training Partner

Trainings

Consulting

Workshops

Puppet Trainings

http://www.netways.de/training

What this talk is all about

certificates

puppet certificates

REST API

distributed environments

security issues and their consequences

certificate lifecyle

WHY SHOULD I CARE?

Running Puppet Enterprise?

CERTIFICATES

Public Key Infrastructure - PKI

everybody has it's own private key

signs or encrypts a message

verification/decryption uses public key

algorithms: RSA, DSA...

PKI - Wikipedia

X.509

describes how our Puppet PKI works

https:// - you use it every day

ITU-T standard

defines a strict hierarchy

a tree instead of a "web of trust"

X509v3: allows extensions

Certificate structure

(distinguished) name

serial number

algorythm

issuer

validity: FROM - TO

...

The distinguished name: DN

just a string

often a DNS name

could also be "CA: puppet master"

something you should care about!

The revocation list

allows to invalidate certificates

does so based on serial numbers

important if you "loose" certificates

Filename extensions

.csr: certificate signing request, Base64

-----BEGIN CERTIFICATE REQUEST-----...-----END CERTIFICATE REQUEST-----

.pem: a certificate, Base64

-----BEGIN CERTIFICATE-----

Puppet uses .pem also for private keys:

-----BEGIN RSA PRIVATE KEY-----

PUPPET CERTIFICATES

Puppet certificates: archeology

Want to see a fresh new Puppet CA? Try it out!

mkdir /tmp/ssltestpuppet master --no-daemonize --verbose \--ssldir /tmp/ssltest \--certname test.example.com

Puppet certificates: archeology

A fresh new Puppet CA!

Puppet certificates: archeology

ls -l /tmp/ssltest

Same thing for the agent

puppet agent --test \--ssldir /tmp/sslagent \--certname test.example.com

We all know the basics

puppet cert list

puppet cert list --all

puppet cert sign test.example.com

puppet cert revoke test.example.com

puppet cert clean test.example.com

find ./ -name 'test.example.com*' --delete

SSL directories

puppet master --configprint ssldirpuppet agent --configprint ssldir

manual configuration makes sense

think about user permissions

~/.puppet, /var/lib/puppet

master and agent on the same host

passenger VS debug (--no-daemonize)

Let's dump a certificate

openssl x509 -in testexample.com.pem -noout -text

puppet cert print test.example.com

Custom data in your certificates

https://docs.puppetlabs.com/puppet/latest/reference /ssl_attributes_extensions.html

/etc/puppet/csr_attributes.yaml

custom attributes in your CSR

MCollective

Study security guidelines!

Study security guidelines!

Study security guidelines!

STUDY SECURITY GUIDELINES!

puppetlabs.com/mcollective/security-overview

Get inspired by existing modules

make sure you understood them

or write your own ones

re-use Puppet certificates

read about trust

and STUDY THE SECURITY GUIDELINES!

THE REST API

It's a web application!

<VirtualHost *:8140>

SSLEngine onSSLProtocol ALL -SSLv2 -SSLv3SSLCipherSuite EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+...SSLHonorCipherOrder on

SSLCertificateFile $ssldir/certs/$fqdn.pemSSLCertificateKeyFile $ssldir/private_keys/$fqdn.pemSSLCertificateChainFile $ssldir/ca/ca_crt.pemSSLCACertificateFile $ssldir/ca/ca_crt.pemSSLCARevocationFile $ssldir/ca/ca_crl.pemSSLVerifyClient optionalSSLVerifyDepth 1SSLOptions +StdEnvVars +ExportCertData

The Rest API

# http://docs.puppetlabs.com/guides/rest_api.html

https://master:8140/{environment}/{resource}/{key}

available on puppet master

and on VERY ancient agents (listen=true)

Puppet REST API URI examples

GET /{environment}/catalog/{node certificate name}

GET /{environment}/file_bucket_file/md5/{checksum}

GET /{environment}/facts/{node certname}

Permissions

http://docs.puppetlabs.com/guides/rest_auth_conf.html

# auth.conf

# allow all nodes to store their own reportspath ~ ^/report/([^/]+)$method saveallow $1

SSL-enabled curl example

Use your certificates and discover the API:

curl \--cert /var/lib/puppet/ssl/certs/host.pem \--key /var/lib/puppet/ssl/private_keys/host.pem \--cacert /var/lib/puppet/ssl/ca/ca_crt.pem \-k -H "Accept: yaml" \https://master:8140/production/facts/somehostname

DISTRIBUTED SETUP

Puppet Advanced* Training

Configuration for such a setup

One CA is more than enough:

[master]ca = false

[agent]ca_server = ca.example.com

Optionally, still experimental: DNS SRV records

Chain of trust

Since 3.2.1 you can use intermediate CAs to delegate trust

# http://docs.puppetlabs.com/puppet/3/reference\# /config_ssl_external_ca.html[agent]ssl_client_ca_auth = $certdir/issuer.pem

Tell Apache about your chain:

SSLCertificateChainFile "/path/to/ca_bundle.pem"

It could look like this

+------------------------+ | Root self-signed CA | +------+----------+------+ | | +----------+ +------------+ | | v v+-----------------+ +----------------+| Master CA | | Agent CA |+--------+--------+ +--------+-------+ | | v v+-----------------+ +----------------+| Master SSL Cert | | Agent SSL Cert |+-----------------+ +----------------+

SSL Professional?

integrate it in your existing hierarchy

use your own toolchain

ship signed certificates (carefully)

SECURITY

Puppet and security issues

Read Security Disclosures!

https://puppetlabs.com/security/

Thank Heartbleed!

docs.puppetlabs.com/trouble_remediate _heartbleed_overview.html

docs.puppetlabs.com/latest/reference /ssl_regenerate_certificates.html

A specific security problem

Very interesting and worth to read: CVE-2011-3872

"In versions prior to 2.6.12 and 2.7.6, the Puppet CA willimproperly insert any certdnsnames values into agentcertificates as well as master certificates. This bug wasintroduced in Puppet 0.24.0."

puppet master --configprint certdnsnamespuppet, puppet.example.com

Study it!

http://links.puppetlabs.com/cve20113872_remediation

Have a look at the remediation toolkit

And to be on the safe side, check your agent certs:

openssl x509 -in test.example.com.pem -noout -text | \grep 'Subject Alt' -A 1

X509v3 Subject Alternative Name: DNS:test.example.com, DNS:puppet, DNS:puppet.example.com

WARNING

"upgrading" doesn't fix a mess like this

old certificates would remain valid

you have to switch to a new CA...

...and this leads us to the next topic

CA LIFECYCLE MANAGEMENT

Bad news

Puppet should allow for automatic resigning of SSL certs

http://projects.puppetlabs.com/issues/7272

There is no such thing in Puppet

"...will be available with Puppet Sites"

YOU ARE ON YOUR OWN

One way of replacing a CA

stop all agents

throw away their certificates

create a new CA with a new name

start your agents

sign their new CSRs

CA...

master: rm -rf $(puppet master --configprint ssldir)agents: rm -rf $(puppet agent --configprint ssldir)

# default ca_name: "Puppet CA: <master certname>"CERTNAME=$(puppet master --configprint certname)TS=$(date +%Y-%m-%d)puppet cert --generate \--ca_name "Puppet CA: $CERTNAME <$TS>" $CERTNAME \--dns_alt_names puppet,puppet.example.com

puppet cert --allow-dns-alt-names sign $CERTNAME

You could also

get inspired by the remediation kits

write your own SSH loop

fix it with MCollective (carefully!)

open new feature requests

Don't like trouble?

Before generating your CA:

[master]ca_ttl = 20y

Leave your company in time

NB: expiration > 2038-01-19 == bad idea

BTW: WE ARE HIRING ;-)

Thank you for your attention!

Questions?

class puppetcamp {

package { 'questions': ensure => answered }

}