Date post: | 28-Jul-2015 |
Category: |
Software |
Upload: | markus-jura |
View: | 91 times |
Download: | 2 times |
Reactive Reference Architecture
Markus Jura @markusjura
Big dataMassive Learning Platform
http://allthingsd.com/files/2013/07/minions-‐640x327.jpg
3
Principles of Reactive Systems
Architecture
4
Web (Play)
backend (Akka)
Journal (Cassandra)
Fast data processing (Spark)
Problem: Splitting application
Monolithic App
6
Splitting App
7
Frontend nodes
Backend nodes
Akka Distributed Pub/Sub• Location transparency
• Publish/Subscribe
• Dynamically add/remove
8
Distributed Pub/Sub Mediator
9
Frontend nodes
Backend nodes
M M
M M M
Code
Frontend
11
// Starting up the mediatorval mediator = DistributedPubSubExtension(system).mediator
// Sending message through mediatormediator ? DistributedPubSubMediator.Send(“/user/course-aggregate”, CreateCourse(course), localAffinity = true)
Backend
12
// Starting up the mediatorval mediator = DistributedPubSubExtension(system).mediator
// Setting up the servicesval courseAggregate = system.actorOf(CourseRepository.props, "course-aggregate")mediator ! DistributedPubSubMediator.Put(courseAggregate)val levelAggregate = system.actorOf(LevelRepository.props, "level-aggregate")mediator ! DistributedPubSubMediator.Put(levelAggregate)
Problem: Distributed State
State on single node
14
Frontend nodes
Backend nodes
State across nodes
15
Frontend nodes
Backend nodes
Akka Cluster Sharding
• Partition actors across Cluster
• Location transparency
• Rebalancing
16
How does it work?
17
How does it work?
18
Code
Backend: Create ShardRegion
20
// Backend: Start cluster sharing in non-proxy modeClusterSharding(system).start( "users", Some(User.props), UserSharding.idExtractor, UserSharding.shardResolver())
Backend: Shard Resolver & Id Extractor
21
// Extract the user shard // Rule of thumb: 10 times the max numbers of sharding nodesdef shardResolver(shardCount: Int = 20) = { case UserService.Envelope(email, _) => email.hashCode % shardCount.toString}
// Extract the id and payload of the user actor val idExtractor: ShardRegion.IdExtractor = { case UserService.Envelope(email, payload) => (email, payload)}
Frontend
22
// Send message via shard region shardRegion ? UserService.Envelope(email, GetUser(email))
// Start sharding in proxy mode on every frontend nodeval shardRegion: ActorRef = ClusterSharding(system).start( "users", None, UserSharding.idExtractor, UserSharding.shardResolver())
Problem: Loosing State
State in memory
24
Frontend nodes
Backend nodes
Persist state
25
Frontend
Backend
Akka Persistence
• Stores state in Journal DB
• Event Sourcing
• Restore state
26
Code
Write user state
28
class User extends PersistentActor {
// User id / sharding extractor id override def persistenceId: String = self.path.name
// Persist UserCreated event to Cassandra journal override def receiveCommand: Receive = { case UserService.CreateUser(userProgress) => persist(UserCreated(userProgress)) { userCreated => // Update user state onUserCreated(userCreated) sender() ! UserService.UserCreated } } ...
Recover user state
29
...
// Replay UserCreated message to recover state override def receiveRecover: Receive = { case e: UserCreated => onUserCreated(e)}
Problem: Replicate Cache
Caching
31
Frontend
Backend
C? C?
CRDT Replicator
32
Frontend nodes
Backend nodes
CR
R R R
RC
What is CRDT?
33
Conflict-free replicated data type is a type of specially designed data structure used to achieve strong eventual consistency
Great talk on CRDT by Sean Cribbs: Eventually Consistent Data Structures
Akka Data replication• CRDT for Akka cluster
• Replicated in-memory data store
• Supports many types: (GSet, ORSet, PNCounter…)
34
Code
Frontend
36
//starting up the replicatorval replicator = DataReplication(system).replicator
//get cache entryval f = replicator ? Get(key, ReadLocal, None)
Backend
37
//starting up the replicatorval replicator = DataReplication(system).replicator
//updating the cachereplicator ! Update(Course.cacheKey, GSet.empty, WriteLocal)
(_ + courseCreated.course)
Q & Option[A]
@markusjura
39
React.js
frontend
REST API Actors
backend
Cache
PubSub CRDTPersistenceCluster Sharding
journal
©Typesafe 2014 – All Rights Reserved