Distributed Software Engineering Middleware for Reconfigurable Systems

Distributed Software Distributed Software EngineeringEngineering

Middleware for Reconfigurable Middleware for Reconfigurable SystemsSystems

Gianpaolo CugolaDipartimento di Elettronica e Informazione

Politecnico di Milano, [email protected]

http://www.elet.polimi.it/~cugola

joint work withGian Pietro Picco, Paolo Costa, Davide Frey,

Matteo Migliavacca, and Amy Murphy

IS-MANET kick-off meeting 2

Politecnico di Milano

OutlineOutline

Sources of reconfiguration Publish-subscribe middleware (Jedi)

– Managing reconfigurations in distributed publish-subscribe middleware

– Reducing event loss during reconfigurations by using epidemic algorithms

State-based, peer-to-peer middleware (PeerWare)– A model combining state based and reactive

communication– A middleware which adopts such model

Future work



Sources of reconfigurationSources of reconfiguration

Modern distributed applications must be able to operate in dynamic environments

At the network level, dynamicity comes from:– Organizational changes in companies, which results

in changes in the company network• New departments, merging among companies, …

– Mobile computing• Nomadic users, base-station, and ad-hoc scenarios

– Peer-to-peer networking• Peers come and go during the application lifetime

Building reconfigurable systems is a challenge for software engineering



Middleware for Middleware for reconfigurable systems - 1reconfigurable systems - 1 To efficiently build reconfigurable applications

developers need new middleware– Applications must be able to reconfigure themselves...– ... and cannot rely on common notions like the address of

remote components

Standard C/S middleware, like RMI and CORBA, present several drawbacks– Being based on a strong coupling among clients and

servers...– ... which reduces the possibility of dynamically changing

the configuration of the application to cope with changes in connectivity

– Even if some services have been added to the above mentioned middleware to try to overcome these problems



Middleware for Middleware for reconfigurable systems - 2reconfigurable systems - 2 In the last years, different middleware have

shown their benefit in a dynamic environment– Publish-Subscribe middleware– State-based, Linda like, middleware

The SE group at Politecnico di Milano has gained a good experience with both– Jedi: A publish-subscribe middleware for large scale

applications– Lime: A state-based middleware based on the Linda

paradigm, explicitly conceived for mobile computing

Recently we got even further by extending Jedi and developing PeerWare, a new state-based, p2p middleware



Publish-Subscribe Publish-Subscribe MiddlewareMiddleware Communication is based on asynchronous event

notifications, published by clients Clients subscribe to event classes by issuing a

subscription Client components are attached to an event

dispatcher, responsible for dispatching events to subscribers

Publish-subscribe middleware varies in terms of:– Event format & subscription language– Architecture

Dispatcher

C C CC

Temperature > 20 oC

Temperature = 25 oC



Subscription LanguageSubscription Language

The expressivity of the subscription language allows to distinguish between:– Subject-based

• The set of subjects is determined a priori• Analogous to multicast

– Content-based• Subscriptions contain expressions (event patterns)

that allow clients to filter events based on their content

• The pattern set is determined by client subscriptions• A single event may match multiple subscriptions



Architecture of the event Architecture of the event dispatcherdispatcher Most publish-subscribe middleware adopt a centralized

event dispatcher To improve scalability and performance, most advanced

publish-subscribe middleware offer a distributed event dispatcher– Where a set of interconnected dispatching servers

cooperate to distribute events Distributed publish-subscribe middleware differ in the

way such dispatching servers are connected ad in the way subscriptions are propagated



Subscription ForwardingSubscription Forwarding

Every dispatcher propagates subscriptions to the others– Subscriptions are never sent twice through the same link

Events follow the routes laid by subscriptions



ReconfigurationReconfiguration

To operate in dynamic environments, distributed publish-subscribe middleware must provide mechanisms to reconfigure the event dispatching network– No publish-subscribe system to date deals with this kind of

reconfiguration Supporting reconfiguration implies solving three

problems:– How to reconstruct the overlay network

• Open issue… to be solved in is-manet

– How rearrange the subscription information• Content-based makes the problem very different from existing

subject-based solutions, inspired by multicast

– How to minimize event loss during reconfiguration• Epidemic algorithms

We considered the three problems as orthogonal



Removing a linkRemoving a link

The graph is partitioned in two Unsubscriptions are propagates along the two

subgraphs, starting at the broken link

A B

C

D



Adding a linkAdding a link

The two subgraphs are merged back into one Subscriptions are propagated across the new link

A B

C

D



DrawbacksDrawbacks

In the case of a link substitution, the ordering of operations affects the efficiency of the reconfiguration

All the subscriptions in the left subtree are removed, but the blue ones are reinserted immediately after

A B

C

D



Understanding PropagationUnderstanding Propagation

A subscription is propagated following the unique route up to the closest splitter, if it exists; to the whole tree, otherwise– A splitter is either a subscriber or a dispatcher whose

routing table has two or more entries for the given subscription

A B

C

D

Adding a subscription is usually cheap

The more splitters, the shorter the path a subscription must follow



The IdeaThe IdeaTo appear in Int. Conf. on Distributed Computing Systems To appear in Int. Conf. on Distributed Computing Systems (ICDCS03)(ICDCS03)

The idea is to perform unsubscriptions first, and subscriptions after– This way, the network is kept dense of subscriptions, and

the impact of reconfiguration is reduced

Unsubscriptions are managed through a timer, which is set when a link breakage is detected

Other optimizations are provided, e.g., for the case where a dispatcher belongs to a broken link and a new link

The idea is very simple but:– It is based on a thorough understanding of the

dispatching strategy– It provides significant improvements



Simulations: Event DeliverySimulations: Event Delivery

Goal: understand whether the algorithm converges, even under a barrage of reconfigurations

non-overlapping overlapping



Simulations: Overhead Simulations: Overhead ReductionReduction

Better performance with a sparse graph The impact of misrouted events is relevant



Reducing event loss during Reducing event loss during reconfigurationreconfiguration

We need a mechanism to reduce event loss during reconfiguration– It should be scalable, light, and resilient to

reconfigurations

The idea: why not try to apply epidemic (gossip) algorithms to recover lost messages?

Epidemic algorithms:– Each process periodically

communicates its view of thesystem state to a random subsetof the other processes

– Information propagates “epide-mically” up to a stable state



Gossip & reconfigurable Gossip & reconfigurable publish-subscribe publish-subscribe middlewaremiddleware Content-based routing introduces new problems Issues to solve

– How to route gossip messages?• Content-based routing schema do not have any notion of

“group”

– How to detect event loss?• Simple numbering schemas are not enough

Different approaches– Push– Pattern-based pull– Source-based pull

Same approaches can be used to recover event lost during reconfigurations or because of channel failures



SimulationsSimulationsOverlapping reconfigurationsOverlapping reconfigurations



SimulationsSimulationsUnreliable channelsUnreliable channels



SimulationsSimulationsDelivery rate vs. sizeDelivery rate vs. size



SimulationsSimulationsOverhead Overhead (gossipMsgs/events)(gossipMsgs/events)



PeerWare: Some facts and an PeerWare: Some facts and an ideaidea

Some facts– Sometimes it is necessary to have state based

communication...– ... other times messages are the best form of

communication– As demonstrated by applications like gnutella, peer-to-

peer architectures perfectly fit the needs of flexibility and reconfigurability of several classes of distributed applications

The idea– Merge in a single, integrated model the two forms of

communication• A model that could be implemented by a scalable middleware

– Develop a peer to peer middleware that adopts this model



PeerWare: The general PeerWare: The general modelmodel Based on the notion of global virtual data

structure– Each mobile unit carries a piece of a global data

structure, e.g., a subtree, a subset, a submatrix, …– When connectivity is available, the portion of the global

data structure available to the unit is larger, and determined by the other connected units

Middleware carries the burden of maintaining the illusion of local access to a global data structure

Wide (and largely unexplored) range of design choices– Rules for breaking/merging the data structure, proactive

vs. reactive primitives, definition of connectivity, …



PeerWare: The specific PeerWare: The specific modelmodel

PeerWare maintains a gvds organized as a graph of nodes and documents– Each node may be part of zero or one “parent”

nodes– Each document may belong to one or more

nodes

Each component connected to PeerWare contributes the gvds with a set of connected nodes and documents

The gvds managed by PeerWare is built by “merging” these documents and nodes



PeerWare: EngagementPeerWare: Engagement

Nodes with the same identifier(i.e., path) are merged together

n1

n2 n3 d1d3

n7

n8 n10 d7

d8n9

d6

Component 1

n1

n2 n3 d1d2

n4 n5

n6 d5

d4

Component 2

n7

n8 n10 d7

d8n9

d6

n1

n2 n3d1@c1d2

n4 n5

n6 d5

d4

d3 d1@c2

dds



PeerWare: PrimitivesPeerWare: Primitives

PeerWare provides primitives to:– Join and leave the system– Manipulate the local repository of each

component• i.e., the set of nodes and documents held by each

component

– Query the distributed data structure– Register subscriptions to events occurring

on data items– Publish event notifications for events

related to data items



PeerWare: Global primitivesPeerWare: Global primitives

Item[] execute(NodeFilter, DataFilter, Action, Callback, Mode)– The query is distributed to the set of hosts holding at

least one node matching NodeFilter– Locally, the set M of items that belong to these nodes

and match the DataFilter is determined– M is passed (locally) to the user-provided Action that

manipulates it (e.g., by filtering out part of their content)

– The set of items returned by the action is given back to the caller• Through the Callback if the asynchronous mode is specified• As a single array of data if the synchronous mode is

specified




SubscriptionID subscribe(NodeFilter, DataFilter, EventFilter, Callback)– Events matching NodeFilter,...– ... occurring on a data item matching DataFilter,...– ... and belonging to a node matching NodeFilter...– ... are returned to the caller through the specified

Callback




<Item[],SubscriptionID> executeAndSubscribe( NodeFilter, DataFilter, EventFilter, Action, Callback, Mode)– Equivalent to an execute atomically followed (on each

single host) by a subscribe– It allows programmers to “hook” upon some data and

provide “strong consistency” on the content of that data• Through event notifications that inform the caller of any

further change on that data



PeerWare: Different PeerWare: Different scenariosscenarios Peer to peer with base stations

Dynamic fringe of mobile or disconnected nodes

Fixed backbone of PeerWare servers



PeerWare: Different PeerWare: Different scenariosscenarios “Strongly” peer to peer

Mobile hosts connects in an ad-hoc network

Querying and publishing/subscribing proceed as before



PeerWare: The prototypePeerWare: The prototype

Current prototype is oriented to enterprise networks– Base station scenario

Provides limited mechanisms to reconfigure the overlay network of peers– Only leaf nodes may join and leave at run-

time



PeerWare: ConclusionsPeerWare: Conclusions

PeerWare brings together the best of two worlds...– It is possible to mix together message-based and state-

based communication• In a single model (with nodes acting as a unique mechanism to

limit the scope of queries and subscriptions)

– It does not suffer of the scalability problems of Linda-like middleware• Access to the distributed data structure is not atomic• Events combined with the executeAndSubscribe primitive

provide great expressive power

... in a peer to peer architecture Applicability

– Within the EU project MOTION PeerWare has been used to develop a collaborative application in an enterprise scenario



Future workFuture work

Publish-subscribe middleware– Find some mechanisms to maintain the

overlay network– Generalize the approaches to other

architectures (generic graphs)– Introduce these mechanisms in Jedi

PeerWare– “Port” the mechanisms above to PeerWare

to support more dynamic scenarios– Validate the middleware by using it in

different environments

Date post:	06-Jan-2016
Category:	Documents
Upload:	brigid
View:	43 times
Download:	0 times

Distributed Software Engineering Middleware for Reconfigurable Systems

Documents