Date post: | 27-Nov-2014 |
Category: |
Software |
Upload: | yasuki-okumura |
View: | 1,182 times |
Download: | 1 times |
Job Queue in B2Bwith Akka
long versionYasuki Okumura
● Shanon, Inc. - Marketing company
● Scala developer for 2 years
twitter: @busterdayogithub: https://github.com/buster84
About me
Shanon hiring Scala developers
● Job Queueing manager● https://github.com/Shanon/Malba
Malba
● Most problematic thing in B2B is job order.● Most of B2B service cases, normal FIFO or
priority Queue don’t fit their business model.
● We need our own Custom Queue.
Motivation
Example - Email service
1. A attempt to send 100,000 emails.2. B attempt to send 100 emails.
Does B have to wait until finishing to send 100,000 emails?
Balancing Queue
A’s 100,000 emails
FIFO
B’s 100 emails
1
32
4
100,001
100,100
Balancing Queue
1 2
100,100
200
4
199
3
● Breadth-first search -> FIFO Queue● Depth-first search -> FILO Queue● A* search -> Priority Queue
Inspired by Graph search
What makes difference among the algorithms is Queue.
● Premium users can send mails faster than normal users.
● It is guaranteed that sending less than 1,000 mails finish in 1 hour.
Other cases
How we use
1. AddTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L9-L17
2. AddWorkerhttps://github.com/Shanon/Malba/blob/master/Malba-
protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L151-L158
3. GetTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L56-L63
Structure + Protocol
Demo
https://github.com/typesafehub/activator-akka-distributed-workershttp://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
You can compare Malba with above.
Strongly inspired by
● CI environment● Make and Publish Packages● Support HTTP● Tests● Documents● Take snapshots● GUI - using PersistentView
TODO
We welcome your PR or comments.https://github.com/Shanon/Malba
Thank you