Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | scott-bennett |
View: | 222 times |
Download: | 0 times |
ZooKeeper: A Distributed Coordi-nation Service for Distributed Ap-
plications
Motivation• Large-scale distributed application
require different forms of coordina-tion:– Configuration– Group membership and leader election– Synchronization
Related Works• Amazon Simple Queue Service-queu-
ing• [25]-leader election• [27]-configuration• Chubby[6]-locking service with
strong synchronization
Zookeeper• Zookeeper is a distributed service for
distributed applications. It support:– Synchronization– Configuration management– Naming service
Why Do We Need Zookeeper• Zookeeper is simple• Zookeeper is replicated• Zookeeper is ordered• Zookeeper is fast
Data Model
• Regular Znode• Ephemeral Znode• Sequential flag
Watches
Zookeeper
Client1
Client4
crea
te/ex
ist/
(WAT
CH)
setDataNOTI
FICA
TION
Client API• create(path, data, flags)• delete(path, version)• exists(path, watch)• getData(path, watch)• setData(path, data, version)• getChildren(path, watch)• sync(path)
Zookeeper Service Architec-ture
• Read request is handled by local server• Write request is sent to the leader, the leader
broadcasts the change to the Zookeeper through Zab an atomic broadcast protocol.
Setup Zookeeper• Download:
http://www.apache.org/dyn/closer.cgi/zookeeper/
• Configure Zookeeper:– Standalone Mode– Replicated Mode
Standalone Mode• create file zoo.cfg with the content:
• Start server: bin/zkServer.sh start• Test with zookeeper client: bin/zk-
Cli.sh -server 127.0.0.1:2181
tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181
Standalone Mode (2)• ls /• get• Set
• …
Setup Zookeeper: Replicated mode
• Every server has the same configura-tion file.
• Create file named myid In the datadir directory. The content of myid file is an unique number. tickTime=2000
dataDir=/home/sdn/zookeeperclientPort=2181initLimit=5syncLimit=2server.1=192.168.0.94:2888:3888server.2=192.168.0.59:2888:3888…server.n=192.168.0.59:2888:3888
Use cases• Naming service• Configuration management• Synchronization• Message Queue• Notification system
Synchronization: Simple Lock• Client has smallest number have permission to access locked object• When the client finishes work with object, child node is deleted.
Another client has smallest number have permission to access locked object
App-Root
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
Client1 Client
2Client
3
Client4
N=crea
te(P,
EPHEM
ERAL
|SEQ
UEN-
TIAL)
N=
crea
te(P
, EP
HEM
ERAL
|SEQ
UEN
-TI
AL)
N=
create(P,
EPHEM
ERAL|SEQUEN
-
TIAL)
N=create(P,
EPHEMERAL|SEQUEN-
TIAL)
Synchronization: Simple LockCheck existing of approot and create it.
Create child node with Sequential and Ephemeral flag And receive a number.
When receive the notification, check whether the number is smallest of child nodes number
Synchronization: Barrierpublic class SimpleLock implements Watcher
ZooKeeper zooKeeper = new ZooKeeper("192.168.0.94:2181", 3000, this);Stat res = zooKeeper.exists(root, true);if(res==null) String abc = zooKeeper.create(root, null, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
String createRes = zooKeeper.create(root+"/", null, Ids.OPEN_ACL_UNSAFE, CreateMod-e.EPHEMERAL_SEQUENTIAL);int number = Integer.parseInt(createRes.substring(root.length()+1));
while (true) {synchronized (root) {
//TODO check whether the number is smallest if(smallest) {dost();}else{root.wait();} }}
public void process(WatchedEvent event) { //Watcher eventsynchronized (root) {root.notify();}
}
private void dost(){System.out.println("Access System.out at "+System.currentTimeMillis());Thread.sleep(5000);
}
Synchronization: Simple Lock
public static void main(String[] args){new SimpleLock();new SimpleLock();new SimpleLock();new SimpleLock();
}
/simpleLock/0000000009:9Access System.out at 1416028020742/simpleLock/0000000010:10Access System.out at 1416028025765/simpleLock/0000000011:11Access System.out at 1416028030779/simpleLock/0000000012:12Access System.out at 1416028035806
Synchronization: Barrier• Every client creates child node of approot.• Whenever the number of child nodes is
enough, client will start work. AppRoot
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
Client1 Client
2Client
3
Client4
N=crea
te(P,
EPHEM
ERAL
|SEQ
UEN-
TIAL)
N=
crea
te(P
, EP
HEM
ERAL
|SEQ
UEN
-TI
AL)
N=
create(P,
EPHEM
ERAL|SEQUEN
-
TIAL)
N=create(P,
EPHEMERAL|SEQUEN-
TIAL)
Synchronization: BarrierCheck existing of approot and create it.
Create child node with Sequential and Ephemeral flag And receive a number.
When receive the notification, check whether the number is smallest of child nodes number
Synchronization: Barrierpublic static void main(String[] args){
new Barrier(3).run();}
Run 3 instances:
/barrier/0000000022Starting at 1416048008944
/barrier/0000000023Starting at 1416048008947
/barrier/0000000024Starting at 1416048008948
Message Queue• Every client creates child node of approot.• Whenever the number of child nodes is
enough, client will start work.AppRoot
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
sender1
sender2
re-ceiver
N=create(P, SEQUEN-TIAL)
N=create(P, SEQUEN-
TIAL)
get(smallestP)
Message Queuepublic class MessageQueue implements Watcherpublic static class Sender extends MessageQueue implements Runnablepublic static class Receiver extends MessageQueue implements Runnable
public void sendMessage() { //Watcher eventString sendMessage = "sendMessage at "+System.currentTimeMillis();zooKeeper.create(root+"/", sendMessage.getBytes(), Ids.OPEN_ACL_UNSAFE, Create-
Mode.PERSISTENT_SEQUENTIAL);}
public void process(WatchedEvent event) { //Watcher eventsynchronized (root) {root.notify();}
}
while (true) {synchronized (root) { List<String> childs = zooKeeper.getChildren(root, true); if(childs.size()==0){root.wait();}else{ //TODO sort the list
for (String child : childs) { byte[] data = zooKeeper.getData(root+"/"+child, false, new Stat()); System.out.println("readMessage:"+new String(data)); zooKeeper.delete(root+"/"+child, 0);} } }}
Message Queue:
Run 2 sender instances and 1 receiver instance:
readMessage:sendMessage at 1416066118908readMessage:sendMessage at 1416066118916readMessage:sendMessage at 1416066118926readMessage:sendMessage at 1416066118929readMessage:sendMessage at 1416066118939readMessage:sendMessage at 1416066118942readMessage:sendMessage at 1416066118952readMessage:sendMessage at 1416066118954readMessage:sendMessage at 1416066118964readMessage:sendMessage at 1416066118967readMessage:sendMessage at 1416066118976readMessage:sendMessage at 1416066118984readMessage:sendMessage at 1416066118989readMessage:sendMessage at 1416066118997readMessage:sendMessage at 1416066119001readMessage:sendMessage at 1416066119009readMessage:sendMessage at 1416066119016