Date post: | 21-Jan-2016 |
Category: |
Documents |
Upload: | david-webb |
View: | 212 times |
Download: | 0 times |
Looking into the Past: EnhancingLooking into the Past: EnhancingMobile Publish/Subscribe MiddlewareMobile Publish/Subscribe Middleware
Exponent:Exponent:Pablo E. GuerreroPablo E. Guerrero
Thesis Supervisor:Thesis Supervisor:Mariano A. CiliaMariano A. Cilia
Universidad Nacional del Centro de la Provincia de Buenos Universidad Nacional del Centro de la Provincia de Buenos AiresAires
Facultad de Ciencias ExactasFacultad de Ciencias Exactas
Tandil, Argentina, Tandil, Argentina, OctoberOctober, 2004, 2004
22
AgendaAgenda
MotivationMotivation Problem StatementProblem Statement Proposed ApproachProposed Approach Design & ImplementationDesign & Implementation Experimental ResultsExperimental Results ConclusionsConclusions
33
MotivationMotivation
Convergence of TechnologiesConvergence of Technologies Usage of Large-Scale Distributed SystemsUsage of Large-Scale Distributed Systems Mobile Devices & Wireless TechnologiesMobile Devices & Wireless Technologies Client/Server (Request/Reply) PredominanceClient/Server (Request/Reply) Predominance
New Paradigm for Information-Driven ApplicationsNew Paradigm for Information-Driven Applications WWW, Sensor Networks, Enterprise StrategiesWWW, Sensor Networks, Enterprise Strategies Pull-based (Request/Reply) Does Not FitPull-based (Request/Reply) Does Not Fit
Polling implies resource wastePolling implies resource waste Leads to Network Saturation or Server Breakdown!Leads to Network Saturation or Server Breakdown! Not Suitable for Mobile DevicesNot Suitable for Mobile Devices
Push-basedPush-based Event-based DisseminationEvent-based Dissemination
Initiator of InteractionInitiator of Interaction
ConsumerConsumer ProducerProducer
Knowledge about Knowledge about CounterpartCounterpart
FullFull Request/Request/ReplyReply
One-to-One One-to-One MessagingMessaging
NoNo Anonymous Anonymous Request/ReplyRequest/Reply
Event-Based Event-Based DisseminationDissemination
44
Motivation (II)Motivation (II)
Publish/Subscribe Middleware providesPublish/Subscribe Middleware provides Asynchronous communicationAsynchronous communication Loose couplingLoose coupling Producers don’t know who is subscribedProducers don’t know who is subscribed Clients can “dynamically” join and leaveClients can “dynamically” join and leave Doesn’t require publishers and subscribersDoesn’t require publishers and subscribers
at the same timeat the same time Reflects intrinsic behavior of Information-Reflects intrinsic behavior of Information-
driven Applicationdriven Application Support for roaming clientsSupport for roaming clients Notion of LocationNotion of Location Location TransparencyLocation Transparency
Pub/Sub needs an Intermediator Pub/Sub needs an Intermediator
55
Publish/Subscribe Systems StructurePublish/Subscribe Systems Structure
Independent Process Data Flow StyleIndependent Process Data Flow Style Message Manager or Notification ServiceMessage Manager or Notification Service Clients (producers/consumers/both)Clients (producers/consumers/both) Communication linksCommunication links
Service SemanticsService Semantics ImplementationImplementation
Decentralized brokersDecentralized brokers Connected graphConnected graph Client’s proxiesClient’s proxies
:aConsumer1 :aProducer1 :aConsumerNProducer
:aProducer3:aConsumer2 :aProducer2
notification service boundary
:aBroker1
:aBroker2
:aBroker3:aBroker4
home stub home stubhome stub
home stub home stubhome stub
:aPublishSubscribeSystem
66
Problem StatementProblem Statement
Problem Found Problem Found @@ Applications Startup Applications Startup Initial Bootstrapping Sequence phaseInitial Bootstrapping Sequence phase Getting a Consistent State Getting a Consistent State
Request/Reply (combination)Request/Reply (combination) Observation phase (pure Pub/Sub)Observation phase (pure Pub/Sub)
Cannot assume how long will it takeCannot assume how long will it take
Meaningful Event
EvntEvntEvntEvntEvnt:anEventBasedApp Sub Evnt:aNotificationService
77
Problem Statement (II)Problem Statement (II)
Problem Aggravated in Mobile Problem Aggravated in Mobile EnvironmentsEnvironments DisconnectionsDisconnections Context Changes (new location)Context Changes (new location) Pervasive environment’s dynamic behaviorPervasive environment’s dynamic behavior
A new bootstrapping phase must be initiated!A new bootstrapping phase must be initiated!
System responsiveness degradedSystem responsiveness degraded
88
Proposed ApproachProposed Approach
Goal:Goal: Reduce bootstrapping latencyReduce bootstrapping latency Use already published (past) notificationsUse already published (past) notifications
Extend an existing pub/sub system, RExtend an existing pub/sub system, REBECAEBECA Store notifications in its InfrastructureStore notifications in its Infrastructure Caches Distributed in the NetworkCaches Distributed in the Network Minimize underlying system changesMinimize underlying system changes
Meaningful Event
Sub EvntEvnt Evnt Evnt Evnt Evnt:aNotificationService:anEventBasedApp
99
The RThe REBECAEBECA Notification Service Notification Service:aConsumer
:anAppObjec t1 :anAppObjec t2
:anAppObjec t3
:aBroker1
:aProducervirtual l ink
notification service boundary
sock
et c
onn
ect
ion
:anEventBroker :aRoutingEngine
so
cket co
nn
ectio
n
home stub layer
applicat ion layer
subscribe(F)
process(E)
:a nEve ntR ou ter1 :aR ou ti ng Eng in e
:aBroker2
:a nEve ntR ou ter2 :aR ou ti ng Eng in e
:aBroker3
:a nEve ntR ou ter3 :aR ou ti ng Eng in e
socket connection
socket connection
1010
RREBECAEBECA Brokers Network Brokers Network
clients
local event brokersX1
border brokers(event routers)
broker’s network
inner broker
routing tables
----
LB1X2----LB2
X4
LB4
X5
LB5
X3---- LB3
X6----LB6
X7----LB7
X8----LB8
ER1
----
----
----
ER5
----
ER4
----
ER3
----
ER2
----
1111
RREBECAEBECA Routing Algorithms Routing Algorithms
FloodingFlooding Filter ForwardingFilter Forwarding
Simple RoutingSimple Routing Identity-based RoutingIdentity-based Routing Covering-based RoutingCovering-based Routing Merging-based RoutingMerging-based Routing
Routing with AdvertisementsRouting with Advertisements
1212
ProposedProposed ApproachApproach
Approach AnalysisApproach Analysis Caching RequirementsCaching Requirements Caching in Event-Based SystemsCaching in Event-Based Systems Notifications StorageNotifications Storage Cache Querying StrategiesCache Querying Strategies GuidelinesGuidelines
1313
Proposed Approach Analysis Proposed Approach Analysis
Store Notifications in Distributed CachesStore Notifications in Distributed Caches Extending External BehaviorExtending External Behavior
Past BoundPast Bound Interface ExtensionInterface Extension
+ void subscribe(Subscription s, PastBound past, EventP rocessor proc)new!
EventBroker
...
+ void advertise(Advertisement a, EventProcessor proc)+ void unadvertise(Advert isement a)+ void subscribe(Subscript ion s, EventProcessor proc)+ void unsubscribe(Subscript ion s)+ void publish(Event e)
1414
Proposed Approach Analysis (II) Proposed Approach Analysis (II)
:aConsumer :aBroker:aCachingStrategy
1.1: doSomething(..)
1.2.1: notify(aCachedEvent1)
1.3: notify(aNewlyPublishedEvent1)
1.4: notify(aNewlyPublishedEvent2)
1: subscribe(aSubscription, aBound, this)
1.2.2: notify(aCachedEvent2)
1.2.3: notify(aCachedEvent3)
Extend Internal BehaviorExtend Internal Behavior Extend System BehaviorExtend System Behavior
Support for Different StrategiesSupport for Different Strategies
1515
Caching Strategies RequirementsCaching Strategies Requirements
Response TimeResponse Time Time OverheadTime Overhead Space UsageSpace Usage IntegrabilityIntegrability
CorrectnessCorrectness TransparenceTransparence DynamismDynamism
Separation of ConcernsSeparation of Concerns Specification of the RequirementsSpecification of the Requirements
from the Infrastructurefrom the Infrastructure
1616
Caching in Event-Based SystemsCaching in Event-Based Systems
Caching applied in a different fashion Caching applied in a different fashion than traditional systemsthan traditional systems
Main differences:Main differences: Cache Coherency and ConsistencyCache Coherency and Consistency Cache SizesCache Sizes Information LifetimeInformation Lifetime Caching vs. BufferingCaching vs. Buffering
1717
Notifications StorageNotifications Storage
When? Locality Principle not Applicable!When? Locality Principle not Applicable! Distributed decision based on:Distributed decision based on:
Subscriptions and UnsubscriptionsSubscriptions and Unsubscriptions Advertisements and UnadvertisementsAdvertisements and Unadvertisements NotificationsNotifications
How? Behaving as FIFO queuesHow? Behaving as FIFO queues Circular Ring BufferCircular Ring Buffer
1818
Notifications Storage (II)Notifications Storage (II)
Parallel Circular Ring BuffersParallel Circular Ring Buffers
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
F1
F2
F5
F4
F3
Take advantage of Content-based NSTake advantage of Content-based NS Filter Identity, Covering and MergingFilter Identity, Covering and Merging
1919
Notifications Storage (III)Notifications Storage (III)
A Combination, whichA Combination, which avoids Replicating Notificationsavoids Replicating Notifications allows Ordering Notificationsallows Ordering Notifications
F1
F2
- -
F3
01
00
01
02
0001 000000000000000101020101020101010101
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
F1
F2
- -
F3
01
00
01
02
0001 000000000000000101020101020101010101
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
MaxGlobalBuffers
MaxFilterBuffers
Ma
xInd
exed
Filt
ers
2020
Cache QueryingCache Querying
NS is an Undirected Acyclic GraphNS is an Undirected Acyclic Graph How deep to search? (i.e. How many levels How deep to search? (i.e. How many levels kk?)?) Link traversals involve costs similar to RPC’s:Link traversals involve costs similar to RPC’s:
De / Serialization, Un / MarshallingDe / Serialization, Un / Marshalling RPC Runtime Costs, Physical Network TransferRPC Runtime Costs, Physical Network Transfer
RPC requires 10RPC requires 10..000~15000~15..000 machine instructions000 machine instructions LPC 100 times faster than RPCLPC 100 times faster than RPC
Strategies Classification:Strategies Classification: k k = 1 = 1 Restricted to Local Event Brokers Restricted to Local Event Brokers 1 < k 1 < k m m Restricted to maximum depth ‘m’ Restricted to maximum depth ‘m’ k k = = Allowed to perform exhaustive search Allowed to perform exhaustive search
2121
Cache Querying: LocalBrokerCachingCache Querying: LocalBrokerCaching
X1
broker’s network
----
LB1X2---- LB2
X4
LB4
X5
LB5
X3---- LB3
X6----LB6
X7----LB7
X8----LB8
ER1
----
----
----
ER5
----
ER4
----
ER3
----
ER2
----
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ngLo ca l
C ac h i ng
Lo ca l
C ac h i ng
2222
Cache Querying: BorderBrokerCachingCache Querying: BorderBrokerCaching
X1
broker’s network
----
LB1X2---- LB2
X4
LB4
X5
LB5
X3---- LB3
X6----LB6
X7----LB7
X8----LB8
ER1
----
----
----
ER5
----
ER4
----
ER3
----
ER2
----
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
Bo rd er
C ac h i n g
2323
Cache Querying: MergingCachingCache Querying: MergingCaching
X1
broker’s network
----
LB1X2---- LB2
X4
LB4
X5
LB5
X3---- LB3
X6----LB6
X7----LB7
X8----LB8
ER1
----
----
----
ER5
----
ER4
----
ER3
----
ER2
----
M e rg i n g
C ac h i n g
M ergingCach in g
M ergingCach in g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M ergingCach in g
M ergingCach in g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M ergingCach in g
M ergingCach in g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M ergingCach in g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
M e rg i n g
C ac h i n g
2424
Cache QueryingCache Querying
Caching w/More Complex Routing AlgorithmsCaching w/More Complex Routing Algorithms Routing Algorithms selects which neighbors to Routing Algorithms selects which neighbors to
forward a subscriptionforward a subscription Solution: Force Forwarding an explicit Solution: Force Forwarding an explicit FetchFetch
messagemessage Caching w/AdvertisementsCaching w/Advertisements
Avoid asking every peer by usingAvoid asking every peer by usingadvertisements informationadvertisements information
Considering the Time DimensionConsidering the Time Dimension Border Brokers’ clocks synchronizationBorder Brokers’ clocks synchronization Attach a Timestamp to notificationsAttach a Timestamp to notifications
2525
Cache Querying GuidelinesCache Querying Guidelines
Floo
din
g
Mer
gin
gR
ou
ting
Co
veri
ngR
ou
ting
Iden
tity
Ro
uti
ng
Sim
ple
Rou
tin
g
Merging Caching
Border Broker Caching
Local Broker Caching
[Cach
ing
Str
ate
gy]
[Routing Algorithm]
[with Advs]
[without Advs]
2626
Design & ImplementationDesign & Implementation
OOPOOPAproachesAproaches ClassClass
InheritanceInheritance DecoratorDecorator
DesignDesignPatternPattern
Flooding SimpleRouting
IdentityRouting
CoveringRouting
MergingRouting
RoutingEngine
CachingEngine
+ decorated)CachingEngine(Engine+ void addSubscript ion( )+ void removeSubscription( )+ void addAdvert isement( )+ void removeAdvert isement( )+ void processEvent( )...
SubscriptionSubscription
AdvertisementAdvertisement
Event
...
MergingCachingEngine ...LocalBrokerCachingEngine
Engine
engine
this.addSubscription(s,ep);
...
+ void addSubscription(Subscription ), EventProcessor
+ void removeSubscript ion(Subscript ion)+ void addSubscription( s, p, ep)Subscription PastBound EventProcessor
+ void addAdvertisement(Advertisement)+ void removeAdvert isement(Advert isement)+ void processEvent(Event)...
new!
2727
AOP Architectural ApproachAOP Architectural Approach
Usage ofUsage ofAOP toolsAOP tools
RoutingEngine
+ void addSubscript ion( )+ void removeSubscript ion( )+ void addAdvertisement( )+ void removeAdvert isement( )+ void processEvent( )# void handleAdminEvent( )...
SubscriptionSubscription
AdvertisementAdvertisement
EventEvent
...
EventBroker
EventRouter
RoutingTable
EventProcessor
CachingStrategy
+ (..)+ void setRout ingEngine( )+ void setSubscriptionEntries( )+ void setAdvertisementEntries( )+ void setProcessors( procs)
CachingStrategyRoutingEngine
RoutingTableRoutingTable
Collection+ void beforeProcessEvent( )Event+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )
+ void beforeUnsubscribe( )
EventEvent
AdminEvent
Subscription
+ void beforeSubscribe( , )+ void afterSubscribe( )
+ void beforeForwardSubsAndUnsubs(..)+ getDestinations( , )
Subscription EventProcSubscription
Set Event EventProcessor
- cache- engine...
EventCacheRoutingEngine
en
gin
e
BorderBrokerCaching ...LocalBrokerCaching
cacheEventCache
...
...
2828
AOP Architectural Approach (II)AOP Architectural Approach (II)
Method InterceptionsMethod Interceptions AdvicesAdvices Inter-Type DeclarationsInter-Type Declarations
AspectsAspects
aRoutingEngine
dispatchnew()
dispatch
dispatch
subscribe (Subscript ion,EventPro cessor)
aLocalEventBroker
aRoutingEngine
addSubscription(Subscription,EventProcessor)
&&
pointcut initRoutingEngine():pointcut initRoutingEngine():initialization(RoutingEngine.new());initialization(RoutingEngine.new());
pointcut subscription():pointcut subscription():withincode(void LocalEventBroker.subscribe(..)) &&withincode(void LocalEventBroker.subscribe(..)) &&call(void RoutingEngine.addSubscription(..));call(void RoutingEngine.addSubscription(..));
before (RoutingEngine routingEngine,before (RoutingEngine routingEngine, Subscription sub,Subscription sub, EventProcessor processor):EventProcessor processor): subscription(routingEngine, sub, processor)subscription(routingEngine, sub, processor){{ if (sub.isBounded())if (sub.isBounded()) routingEngine.getCachingStrategy().beforeSubscribe(sub, processor);routingEngine.getCachingStrategy().beforeSubscribe(sub, processor);}}
public void EventBroker.subscribe(Subscription sub,public void EventBroker.subscribe(Subscription sub, EventProcessor source,EventProcessor source, PastBound bound);PastBound bound);
private PastBound Subscription.bound;private PastBound Subscription.bound;
2929
AOP Architectural Approach (III)AOP Architectural Approach (III)
CachingCachingClassClassHierarchyHierarchy
+ (..)+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )+ void beforeSubscribe( )+ void afterSubscribe( )+ void beforeUnsubscribe( )+ void beforeForwardSubsAndUnsubs(. .)+ getDest inations( , )
LocalBrokerCachingEvent
EventAdminEvent
SubscriptionSubscription
Subscription
Set Event EventProcessor
LocalBrokerCaching
cache
engine
...
RoutingEngine
EventCache
CachingStrategy
+ (..)+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )+ void beforeSubscribe( )+ void afterSubscribe( )+ void beforeUnsubscribe( )+ void beforeForwardSubsAndUnsubs(. .)+ getDest inations( , )# void checkDeliver( , )# void prepareReply( , )# void handleReply( )# getRoutingEntryFor( )
BorderBrokerCachingEvent
EventAdminEvent
SubscriptionSubscription
Subscription
Set Event EventProcessorSubscription EventProcessorSubscription EventProcessor
ReplyRoutingEntry Subscription
BorderBrokerCaching
...
... ...
+ (. .)# void prepareReply( , )# void checkDeliver( , )# void processAdminEvent( )+ void afterProcessEvent( )# void handleReply( )
MergingCachingSubscription EventProcessorSubscription EventProcessor
AdminEventEvent
Reply
MergingCaching
HashTable fetchedReplies
3030
Experimental ResultsExperimental Results
Provide details on Caching BehaviorProvide details on Caching Behavior Evaluations SetupEvaluations Setup
Hierarchical NetworkHierarchical Network Consumers issue RandomConsumers issue Random
SubscriptionsSubscriptions Producers publishProducers publish
Notifications of Random TypesNotifications of Random Types Caching Efficacy MetricCaching Efficacy Metric
Measures Best-EffortMeasures Best-Effort
3131
LocalBrokerCaching EvaluationLocalBrokerCaching Evaluation
3232
BorderBrokerCaching EvaluationBorderBrokerCaching Evaluation
3333
MergingCaching EvaluationMergingCaching Evaluation
3434
Caching Strategies ComparisonCaching Strategies Comparison
3535
ConclusionsConclusions
Notification Service was ExtendedNotification Service was Extended Reduced Bootstrap LatencyReduced Bootstrap Latency Used a pure pub/sub ApproachUsed a pure pub/sub Approach Minimal Changes on the pub/sub Minimal Changes on the pub/sub
Subscription InterfaceSubscription Interface Incorporated Distributed CachingIncorporated Distributed Caching
Efficient, Content-based Notification Efficient, Content-based Notification StorageStorage
Several Strategies Analyzed & Several Strategies Analyzed & ImplementedImplemented
Aspect-OrientationAspect-Orientation Achieved good modularity for a Achieved good modularity for a
crosscutting concerncrosscutting concern Allowed a plug-and-play functionalityAllowed a plug-and-play functionality
EvaluationEvaluation Measured & Compared Caching EfficacyMeasured & Compared Caching Efficacy Gave Insights of Caching BehaviorGave Insights of Caching Behavior
NotificationService
Caching
3636
Future WorkFuture Work
Caching Functionality IntegrationCaching Functionality Integration On-line IntegrationOn-line Integration
through Dynamic AOPthrough Dynamic AOP Experiment with otherExperiment with other
Notification ServicesNotification Services
Extend EvaluationsExtend Evaluations Develop Analytical Response Time Develop Analytical Response Time
ModelsModels Perform Large Scale SimulationsPerform Large Scale Simulations
Consider Other EnhancementsConsider Other Enhancements Priorization and Interactions with Priorization and Interactions with
other Aspects of the Notification other Aspects of the Notification Service Service
NotificationService
Caching