Understanding Container Security

Overview• A Brief History and Overview of Containers• Security Benefits of Containers• Container Vulnerability Management• Responding to Container Attacks

Survey – How familiar are you with containers?• I open them every day – gotta eat to survive• I read about them on TechCrunch• I run them on my raspi at home• We run our production workloads in containers• I contribute code to open source container-related projects

Brief History of Containers

Containers are not new, but…

Container History Timeline

Unix V7

FreeBSD Jails

Solaris Zones

OpenVZ

Process Containers

cgroups

AIX WPARs

LXC

LMCTFY

Docker

1979 2000 2004 2005 2006 2007 2008 2013

How Are Organizations Using Containers?

Container Tech is Being Adopted Quickly

Source: ClusterHQ

Container Security : Top # 3

Container Adoption Challenges

Containers in the Future• Phones• IOT• Maybe cars?

Survey – what container platform do you use?• Docker• LXC• LXD• rkt• Solaris/SmartOS based• Unikernel/microkernel or similar• Why didn’t you list my platform? Everyone uses it!

Brief Overview of Container Orchestration

Why Orchestration?• For “real” workloads:

• How to launch 500 containers across 20 hosts?• Being aware of resources on each host• Getting storage and networking to right container on the right host• Distribution for speed, efficiency, cost, etc.• As part of a CI/CD process

• How to do a rolling update of those 500 live containers to a new sw version?

Lots to Orchestrate

Customer VM

VM Image Management Networking

Customer VM

Local Storage NAS/SAN

Lots to Orchestrate

Customer VM

VM Image Management Networking

Customer VM

Local Storage NAS/SAN

Containers

Container Image mgmt

Container networking

Container storage

Host

Host Image Mgmt

Host Networking

Local Storage NAS/SAN

Lots to Orchestrate

Containers

Container Image mgmt

Container networking

Container storage

Host

Host Image Mgmt

Host Networking

Local Storage NAS/SAN

• Swarm networking• Weave networking• Project Calico networking• CoreOS Flannel networking• Flocker storage• Gluster storage• CoreOS Torus storage• …• ...

We haven’t talked security, yet.

Survey – How Familiar Are You With Information Security?• It’s common for me to get viruses and ransomware• I’m paid to write code by a deadline• I learned my lesson the first time and now try my best• Due to unspecified agreements I cannot answer this question

Security Benefits of Containers and Microservices• Smaller surface area*• Shorter lifespan* – shorter period when open to attack• More automated process – easier to recreate/redeploy*

*(in theory)

Security Benefits of Containers and Microservices• Containerized apps lend themselves to ”12 factor” design

12factor.net

Security Disadvantages of Containers and Microservices• Relatively new technology• Lots of moving parts• Shorter lifespan – this makes investigations more difficult

Container Security Adoption

Survey – What’s your biggest container security concern?• Image security• Host security• Vulnerability management• Container isolation

Results of Twitter Survey

Image Security• Where did an image come from?• Is it an official image?• Is it the right version?• Has somebody modified it?

Image Security• Docker Content Trust

export DOCKER_CONTENT_TRUST=1

• CoreOS image signing and verificationpgp based

Host Security• Follow standard hardening processes (Bastille, Center for Internet

Security, etc.) but only firewall host, not it’s containers• A host itself shouldn’t be “exposed” – there should be no public

attack surface. Administer via known private network

• One nasty exposure – privileged containers.

Vulnerability Management in a Container World

Managing Security Exposure in Containers

Smaller Image, Less Vulnerabilities• Avoid ”From:Debian” and similar

• Software can’t be vulnerable if it’s not installed.

An amazingly large percentage of public Docker images are based on Debian, Ubuntu, or CentOS.

Why? Least Privilege• We want the smallest image possible, when we load it across 100

hosts• The smaller the image, the less exposure for potential vulnerabilities

• If the parent image has a vulnerability, everybody based on that parent has to re-spin their image

Container Vulnerability Scanners• Open Source:

• OpenSCAP• CoreOS Clair• Anchore

• Commercial:• Why go with commercial? Might be easier, packaged.

Vulnerability Triage• Developers are being exposed to the secops work of

vulnerability/patch management

Understand CVSSv2

Understand CVSS Calculator

Container Isolation

Why Isolate?• Only as secure as your weakest link• What happens if other departments are running in your private

cloud?• What happens if other customers are running in your bare metal

CaaS?

CapabilitiesWorst to best:

• Run with --privileged=true

• Run with –cap-add ALL

• Run with --cap-drop ALL --cap-add <only needed>

• Run as non-root user, unprivileged

Useful: capabilities section of https://docs.docker.com/engine/reference/run/

SeccompWe need to build a list of system calls called by the program…

…that we want to succeed

• Guess (preferably educated)• RTFM (thanks John!)• Capture behavior – maybe /usr/sbin/strace• Disassembly?

Plan For Container Attacks• Before going to production, think about how you’d investigate an

attack

• Containers are mostly ephemeral• Collect logs at a central location (ELK, Loggly, etc.)• Practice identifying and snapshotting problem containers• Don’t forget about data backup/recovery

Layered Insight OzoneComprehensive container-native security

Deep visibility and fine-grained control

Automatic behavioral templates

Machine learning based anomaly detection

Layered Insight Ozone

Inside-Out Approach

Workload Portability

No Special Privileges (Userspace)

Zero Impact to Devs / DevOps

Fully Automatic

LI Instrumented Containers

Infrastructure

Host OS

Docker

Thanks – Let’s continue the conversation! @johnlkinsella

https://www.layeredinsight.com

Slides posted at http://www.slideshare.net/jlkinsel

Links• https://docs.docker.com/engine/security/trust/content_trust/ • https://coreos.com/rkt/docs/latest/signing-and-verification-guide.html • https://benchmarks.cisecurity.org/• https://nvd.nist.gov/cvss/v2-calculator

Data Sources

• Moments in Container History: Pivotal• Container Adoption behavior: DataDog• Container Adoption challenges: ClusterHQ• Container Security adoption rates: SDX Central• Layered container image: Ubuntu

Data and some graphics provided by: