+ All Categories
Home > Technology > London HUG 8/3 - Nomad

London HUG 8/3 - Nomad

Date post: 14-Apr-2017
Category:
Upload: london-hashicorp-user-group
View: 493 times
Download: 1 times
Share this document with a friend
73
Nomad HASHICORP
Transcript
Page 1: London HUG 8/3 - Nomad

NomadHASHICORP

Page 2: London HUG 8/3 - Nomad

HASHICORP

Mitchell Hashimoto @mitchellh

Page 3: London HUG 8/3 - Nomad
Page 4: London HUG 8/3 - Nomad

HASHICORP

Page 5: London HUG 8/3 - Nomad

HashiConf 2016June 13-15 Amsterdam

One day of training followed by two days of talks in Amsterdam on all things HashiCorp

HASHICONF.EU >

by HashiCorp

Page 6: London HUG 8/3 - Nomad

NomadHASHICORP

Distributed

Op5mis5cally Concurrent

Scheduler

Page 7: London HUG 8/3 - Nomad

NomadHASHICORP

Distributed

Op5mis5cally Concurrent

Scheduler

Page 8: London HUG 8/3 - Nomad

HASHICORP

Schedulers map a set of work to a set of resources

Page 9: London HUG 8/3 - Nomad

HASHICORP

CPU Scheduler

Web Server -Thread 1

CPU - Core 1

CPU - Core 2

Web Server -Thread 2

Redis -Thread 1

Kernel -Thread 1

Work (Input) Resources

CPU Scheduler

Page 10: London HUG 8/3 - Nomad

HASHICORP

CPU Scheduler

Web Server -Thread 1

CPU - Core 1

CPU - Core 2

Web Server -Thread 2

Redis -Thread 1

Kernel -Thread 1

Work (Input) Resources

CPU Scheduler

Page 11: London HUG 8/3 - Nomad

HASHICORP

Schedulers In the Wild

Type Work Resources

CPU Scheduler Threads Physical Cores

AWS EC2 / OpenStack Nova Virtual Machines Hypervisors

Hadoop YARN MapReduce Jobs Client Nodes

Cluster Scheduler Applica5ons Servers

Human Scheduler Mee5ngs Hours in a Day

Page 12: London HUG 8/3 - Nomad

HASHICORP

Advantages

Higher Resource U5liza5on

Decouple Work from Resources

BeMer Quality of Service

Page 13: London HUG 8/3 - Nomad

HASHICORP

Advantages

Bin Packing

Over-Subscrip5on

Job Queueing

Higher Resource U5liza5on

Decouple Work from Resources

BeMer Quality of Service

Page 14: London HUG 8/3 - Nomad

HASHICORP

Advantages

Abstrac5on

API Contracts

Standardiza5on

Higher Resource U5liza5on

Decouple Work from Resources

BeMer Quality of Service

Page 15: London HUG 8/3 - Nomad

HASHICORP

Advantages

Priori5es

Resource Isola5on

Pre-emp5on

Higher Resource U5liza5on

Decouple Work from Resources

BeMer Quality of Service

Page 16: London HUG 8/3 - Nomad

HASHICORP

Page 17: London HUG 8/3 - Nomad

NomadHASHICORP

Page 18: London HUG 8/3 - Nomad

NomadHASHICORP

Cluster Scheduler

Easily Deploy Applica5ons

Job Specifica5on

Page 19: London HUG 8/3 - Nomad

HASHICORP

example.nomad

# Define our simple redis jobjob "redis" { # Run only in us-east-1 datacenters = ["us-east-1"]

# Define the single redis task using Docker task "redis" { driver = "docker"

config { image = "redis:latest" }

resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis” {} } } }}

Page 20: London HUG 8/3 - Nomad

HASHICORP

Job Specification

Declares what to run

Page 21: London HUG 8/3 - Nomad

HASHICORP

Job Specification

Nomad determines where and manages how to run

Page 22: London HUG 8/3 - Nomad

HASHICORP

Job Specification

Powerful yet simple

Page 23: London HUG 8/3 - Nomad

HASHICORP

# Define our simple redis jobjob "redis" { # Run only in us-east-1 datacenters = ["us-east-1"]

# Define the single redis task using Docker task "redis" { driver = "docker"

config { image = "redis:latest" }

resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10

port “redis” {} } } }}

Page 24: London HUG 8/3 - Nomad

HASHICORP

Containerized

Virtualized

Standalone

Docker

Qemu / KVM

Java Jar

Static Binaries

Page 25: London HUG 8/3 - Nomad

HASHICORP

Containerized

Virtualized

Standalone

Docker

Jetpack

Windows Server Containers

Qemu / KVM

Hyper-V

Xen

Java Jar

Static Binaries

C#

Page 26: London HUG 8/3 - Nomad

NomadHASHICORP

Application Deployment

Docker

Multi-Datacenter and Multi-Region

Flexible Workloads

Bin Packing

HCL Job Specifications

Page 27: London HUG 8/3 - Nomad

NomadHASHICORP

Easy for developers

Operationally simple

Built for scale

Page 28: London HUG 8/3 - Nomad

HASHICORP

Easy for Developers

Page 29: London HUG 8/3 - Nomad

HASHICORP

Nomad for Developers

Simple Data Model

Declarative Job Specification

Sane Defaults

Page 30: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

# Restrict the parallelism in updates update { stagger = “60s” max_parallel = 3 }… }

Page 31: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

group “api” { # Scale our service up count = 5 … }}

Page 32: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

group “api” { # Scale our service down count = 3 … }}

Page 33: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

group “hdfs-data-node” { # Ensure the scheduler does not put # multiple instances on one host constraint { distinct_hosts = true } … }}

Page 34: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

group “hdfs-data-node” { # Attempt restart of tasks if they # fail unexpectedly restart { attempts = 5 interval = “10m” delay = “30s” } … }}

Page 35: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

task “my-app” { # Ensure modern kernel available constraint { attribute = “kernel.version” version = “>= 3.14” } … }}

Page 36: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

task “my-app” { # Inject environment variables env { MY_FEATURE_FLAG = “ON” } … }}

Page 37: London HUG 8/3 - Nomad

HASHICORP

job “foobar” {

task “my-app” { # Register with Consul for service # discovery and health checking service { port = “http” check { type = “tcp” interval = “10s” } } … }}

Page 38: London HUG 8/3 - Nomad

HASHICORP

job “foobar” { # Make sure this task runs everywhere type = “system”

# Nothing should evict our collector priority = 100

task “stats-collector” { … }}

Page 39: London HUG 8/3 - Nomad

Terminal

HASHICORP

$ nomad agent -dev==> Starting Nomad agent...==> Nomad agent configuration:

Atlas: <disabled> Client: true Log Level: DEBUG Region: global (DC: dc1) Server: true

==> Nomad agent started! Log data will stream in below:

[INFO] serf: EventMemberJoin: nomad.global 127.0.0.1 [INFO] nomad: starting 4 scheduling worker(s) for [service batch _core] [INFO] raft: Node at 127.0.0.1:4647 [Follower] entering Follower state [INFO] nomad: adding server nomad.global (Addr: 127.0.0.1:4647) (DC: dc1) [DEBUG] client: applied fingerprints [storage arch cpu host memory] [DEBUG] client: available drivers [docker exec]

Page 40: London HUG 8/3 - Nomad

NomadHASHICORP

Infrastructure As Code

Declarative Jobs

Desired State

Emergent State

Page 41: London HUG 8/3 - Nomad

HASHICORP

Operationally Simple

Page 42: London HUG 8/3 - Nomad

HASHICORP

Client Server

Page 43: London HUG 8/3 - Nomad

HASHICORP

Built for Scale

Page 44: London HUG 8/3 - Nomad

HASHICORP

Built on Experience

gossip consensus

Page 45: London HUG 8/3 - Nomad

HASHICORP

Built on Research

gossip consensus

Page 46: London HUG 8/3 - Nomad

HASHICORP

Single Region Architecture

SERVER SERVER SERVER

CLIENT CLIENT CLIENTDC1 DC2 DC3

FOLLOWER LEADER FOLLOWER

REPLICATIONFORWARDING

REPLICATIONFORWARDING

RPC RPC RPC

Page 47: London HUG 8/3 - Nomad

HASHICORP

Multi Region Architecture

SERVER SERVER SERVERFOLLOWER LEADER FOLLOWER

REPLICATIONFORWARDING

REPLICATION

REGION B GOSSIP

REPLICATION REPLICATIONFORWARDING

REGION FORWARDING

REGION A

SERVERFOLLOWER

SERVER SERVERLEADER FOLLOWER

Page 48: London HUG 8/3 - Nomad

NomadHASHICORP

Region is Isolation Domain

1-N Datacenters Per Region

Flexibility to do 1:1 (Consul)

Scheduling Boundary

Page 49: London HUG 8/3 - Nomad

HASHICORP

Thousands of regions

Tens of thousands of clients per region

Thousands of jobs per region

Page 50: London HUG 8/3 - Nomad

HASHICORP

Optimistically Concurrent

Page 51: London HUG 8/3 - Nomad

HASHICORP

Data Model

Page 52: London HUG 8/3 - Nomad

HASHICORP

Evaluations ~= State Change Event

Page 53: London HUG 8/3 - Nomad

HASHICORP

Create / Update / Delete Job

Node Up / Node Down

Allocation Failed

Page 54: London HUG 8/3 - Nomad

HASHICORP

“Scheduler” =

func(Eval) => []AllocUpdates

Page 55: London HUG 8/3 - Nomad

HASHICORP

Scheduler func’s can specialize

(Service, Batch, System, etc)

Page 56: London HUG 8/3 - Nomad

HASHICORP

Evaluation Enqueue

Page 57: London HUG 8/3 - Nomad

HASHICORP

Evaluation Dequeue

Page 58: London HUG 8/3 - Nomad

HASHICORP

Plan Generation

Page 59: London HUG 8/3 - Nomad

HASHICORP

Plan Execution

Page 60: London HUG 8/3 - Nomad

HASHICORP

External Event

Evalua?on Crea?on

Evalua?on Queuing

Evalua?on Processing

Op?mis?c Coordina?on

State Updates

Page 61: London HUG 8/3 - Nomad

HASHICORP

Server Architecture

Omega Class Scheduler

Pluggable Logic

Internal Coordination and State

Multi-Region / Multi-Datacenter

Page 62: London HUG 8/3 - Nomad

HASHICORP

Client Architecture

Broad OS Support

Host Fingerprinting

Pluggable Drivers

Page 63: London HUG 8/3 - Nomad

HASHICORP

Fingerprinting

Operating System

Hardware

Applications

Environment

Type Examples

Kernel, OS, Versions

CPU, Memory, Disk

Java, Docker, Consul

AWS, GCE

Page 64: London HUG 8/3 - Nomad

HASHICORP

Fingerprinting

Constrain Placement and Bin Pack

Page 65: London HUG 8/3 - Nomad

HASHICORP

Drivers

Execute Tasks

Provide Resource Isolation

Page 66: London HUG 8/3 - Nomad

NomadHASHICORP

Workload Flexibility:

Schedulers

Fingerprints

Drivers

Job Specification

Page 67: London HUG 8/3 - Nomad

NomadHASHICORP

Operational Simplicity:

Single Binary

No Dependencies

Highly Available

Page 68: London HUG 8/3 - Nomad

HASHICORP

Page 69: London HUG 8/3 - Nomad

HASHICORP

Nomad 0.3

Docker, Qemu, Exec, Java, Raw Drivers

Service, Batch, System Schedulers

Periodic Jobs

Service Discovery

Rolling Updates

Job Queueing

Filesystem API (Logging, stdout/stderr, etc.)

Flexible Constraints

Page 70: London HUG 8/3 - Nomad

HASHICORP

Nomad Near Future

Log Shipping

Plugin System

Volumes

Page 71: London HUG 8/3 - Nomad

NomadHASHICORP

Cluster Scheduler

Easily Deploy Applica5ons

Job Specifica5on

Page 72: London HUG 8/3 - Nomad

NomadHASHICORP

Higher Resource U5liza5on

Decouple Work from Resources

BeMer Quality of Service

Page 73: London HUG 8/3 - Nomad

HASHICORP

Thanks! Q/A


Recommended