Cilium: Seattle Kubernetes MeetUp Dec 2017

transcript

Application-Aware Security for Microservices via BPF

Cynthia Thomas, Technology Evangelist@_techcet_

Seattle Kubernetes MeetUpDecember 12th, 2017

Open Source Cloud Native Security

Application Architectures

Delivery Frequency

Operational Complexity

Single Server App

Yearly

Evolution of Application Design & Delivery Frequency

Delivery Frequency

Single Server App

Yearly

3-Tier App

Monthly

Moderate

Delivery Frequency

Single Server App

Yearly

Distributed Microservices

10-100 x’s / day

Extreme

3-Tier App

Monthly

Moderate

Network Securityhas barely evolved

$ iptables -A INPUT -p tcp \-s 15.15.15.3 --dport 80 \-m conntrack --ctstate NEW \-j ACCEPT

The world still runs on iptablesmatching IPs and ports:

Your HTTP ports be like …

Network Securityfor Microservices

Gordon the intern has a brilliant idea…

Gordon wants to build a serviceto tweet out all job offerings.

We’re Hiring!

TweetService

GET /healthz

GET /jobs/{id}

GET /applicants/{job-id}

POST /jobs

APIGET /jobs/{id}

Jobs APIService

TweetService

The Jobs API service has all thedata Gordon needs.

GET /healthz

GET /jobs/{id}

POST /jobs

APIGET /jobs/331

GET /jobs/{id}

Jobs APIService

TweetService

Gordon uses the GET /jobs/ API call

GET /healthz

GET /jobs/{id}

POST /jobs

APIGET /jobs/331

GET /jobs/{id}

TLS Jobs APIService

TweetService

Developer etiquette.Super simple stuff.

Gordon uses mutual TLS AuthGood thinking Gordon

GET /healthz

GET /jobs/{id}

POST /jobs

APIGET /jobs/331

The security team has L3/L4 network security in place for all services

GET /jobs/{id}

Jobs APIService

TweetService

iptables -s 10.1.1.1-p tcp --dport 80-j ACCEPT

Gordon could POST /jobs or GET /applicants(mistakenly or haphazardly).

POTUS job available!

TweetService

Jobs APIService

GET /healthz

GET /jobs/{id}

POST /jobs

exposed

GET /jobs/331

Large parts of the API are still exposed unnecessarily

TweetService

GET /jobs/{id}

iptables -s 10.1.1.1-p tcp --dport 80-j ACCEPT

Not exactlyleast privilegeSecurity

GET /healthz

GET /jobs/{id}

POST /jobs

APIGET /jobs/331

Back to the drawing board…

GET /jobs/{id}

TLS Jobs APIService

TweetService

GET /healthz

GET /jobs/{id}

POST /jobs

APIGET /jobs/331

Least privilege security for microservices

GET /jobs/{id}

FROM“TurtleTweets”ALLOW“GET/jobs/”

TLS Jobs APIService

TweetService

We demanda demo

BPF - TheSuperpowersinside Linux

KubernetesIntegration

NetworkPolicy

StandardResources

L3,L4policy(ingressonlyink8s1.7)

NetworkPolicy

Services

StandardResources

L3,L4policy

ClusterIP,NodePort,LoadBalancer

NetworkPolicy

Services

StandardResources

L3,L4policy

Pods PodLabelstospecifypolicyon

NetworkPolicy

Services

StandardResources

L3,L4policy

NodeIP toNodeCIDRmapping

NetworkPolicy

CiliumNetworkPolicy

Services

StandardResources

CustomResourceDefinitions(CRD)

L3,L4policy

L3(Labels/CIDR),L4,L7(ingress&egress)

NodeIP toNodeCIDRmapping

ShouldIencapsulateornot?

ModeI:Overlay

Name NodeIP Node CIDRNode 1 192.168.10.1 10.0.1.0/24Node 2 192.168.10.8 10.0.2.0/24Node 3 192.168.10.9 10.0.3.0/24

KubernetesNoderesourcestable:

Installation

Run the kube-controller-manager with the --allocate-node-cidrsoption

ModeI:Overlay ModeII:NativeRoutingNode1

L3 Network

Usecase:• Runyourownroutingdaemon• Usethecloudprovider’srouter

Usecase:• Simple• “Justworks”onKubernetes

L3 Policy (Labels Based)

Metadata

Allow frompods

Pods the policyapplies to…

From Pod

To Pod

L3 Policy (CIDR)

Metadata

Allow toIP 8.8.8.8/32

Pods the policyapplies to…

To CIDR

From Pod

L4 Policy

Metadata

Policy appliesto pods …

Allow incomingon port 80

To Port

L4 Policy

Rule 2:Allow PUTIf header is set

Rule 1:Allow “GET /v/1”

L7 Policy – Only allow “GET /v1/”

Allowed API

How are these policies enforced?

• L3 & L4: BPF in the kernel

How are these policies enforced?

• L3 & L4: BPF in the kernel

• L7: Sidecar proxy or KProxy / BPF

Node 2Node 1

ServiceService HTTPRequest

What is a sidecar proxy?

Node 1

Service

SidecarProxy

Node 2

Service

SidecarProxy

Node 1

Service

SidecarProxy

Node 2

Service

SidecarProxy

Node 2Node 1

ServiceService

HTTPRequestSidecarProxy

SidecarProxy

Node 2Node 1

ServiceService

HTTPRequestSidecarProxy

SidecarProxy

Provides L7 functionality• Routing / Load balancing• Retries

• Circuit breaking• Metrics

More info? Google is your friend “sidecar” / “service mesh”

Node 2Node 1

Service

OperatingSystem

Service

Network

SidecarProxy

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

• 3x Socket memory requirement• 3x TCP/IP stack traversals• 3x Context switches• Complexity

Networking Path with a Sidecar

Network

Canweturnthesidecarintoaracecar?

Node 2Node 1

OperatingSystem

Kernel Proxy

Network

Socket

KProxywithBPF

TCP/IP

Socket

TCP/IP

KProxywithBPF

kTLS kTLSSidecarProxy

SidecarProxy

Network

Socket Redirect

Socket Socket

TCP/IP TCP/IP

Loopback

Socket Redirect

Socket Socket

TCP/IP TCP/IP

Loopback

Socket Redirect – Performance?

More info: https://www.cilium.io/blog/istio

Node 2Node 1

Service

OperatingSystem

Service

Network

SidecarProxy

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

The Before and After

Network

Node 1 Node 2

Service

OperatingSystem

Service

Network

Socket

TCP/IP

The Before and After

KProxy

Socket

TCP/IP

KProxy

Network

Cilium Summary• Kubernetes, Mesos, Docker

• CNI / libnetwork

• Networking: Overlay or Native Routing

• Network Security (ingress/egress)

• L3 (Identity or CIDR), L4

• L7: HTTP (0.11), Kafka (0.12), gRPC (0.12)

• Load Balancing (XDP / BPF)

• Dependencies: kvstore (etcd / consul)

Application-Aware Security for Microservices via BPF

@ciliumprojectStar Us on GitHub! http://github.com/cilium/cilium

Thank You! Questions?Tutorial / Getting Started:http://cilium.io/try