● Explicitly (often manually) configured nodes with implicit roles

● Static addresses, often hard-coded

● Scaling up/down means complex re-configuring

● Failure recovery means re-building failed node(s)

● Service discovery isn’t useful

● Explicitly configured roles, automatically configured nodes

● Dynamic addresses, never hard-coded

● Scaling up/down can happen often and automatically

● Failure recovery can be automatic

● Service discovery is crucial

● Strongly consistent and fairly simple

● Relatively easy to maintain● Single point of failure (eww)● Updates propagate only

when instances query

● Strongly consistent but complex

● Fairly difficult to maintain● No single point of failure● Updates propagate only

when instances query

● Eventually consistent but simple

● Fairly easy to maintain● No single point of failure● Updates propagate

automatically, quickly

● Membership - “I exist”○ Nodes maintain entire cluster state and

can execute custom handler scripts when membership changes.

● Failure detection and recovery○ Serf automatically notifies the cluster of

failed nodes. Handler scripts can be used to attempt recovery.

● Event propagation○ Serf can broadcast custom events to the

cluster. These can be used to trigger deploys, propagate configuration, etc.

http://serfdom.io

● With ZooKeeper, doozerd, etcd○ Serf membership and custom events can

trigger updates from a distributed store

● With Chef, Puppet, Salt○ Serf can remove global state from these

tools and allow nodes to join in seconds instead of minutes

● With Fabric, knife○ Serf can provide a list of nodes to execute

on, or redistribute commands via custom events more quickly than an SSH command

http://serfdom.io

● Can (should) live on every instance○ Linux, Mac OS X and Windows. Extremely

lightweight: 5 to 10 MB of memory, primarily communicates with UDP gossip.

● Written in Go○ Fast, embeddable in other Go services

● Open source○ https://github.com/hashicorp/serf

○ Created by Hashicorp, maker of Vagrant (http://hashicorp.com)

○ Mozilla Public License, v2.0

http://serfdom.io

● mDNS / multi-cast for auto-discovery○ Currently a new node needs to know at least

one other node’s address to join the cluster

● Web UI / API with embedded agent○ Monitor the entire cluster, visualize events as

they’re being propagated

○ Request membership and event information from other services via API

● Plugins for other services○ Notify load balancers, DNS, Memcache or

Redis clusters, configuration tools, etc.

http://serfdom.io