1 10/23/98Lunchtime Meeting BBN Technologies Toolkit for Creating Adaptable Distributed Applications...

1 10/23/98 Lunchtime Meeting BBN Technologies

Toolkit for Creating Adaptable Distributed Applications

Joe Loyall, Rick Schantz, Rodrigo Vanegas,

James Megquier, Mark Berman

“Adapt or perish, now as ever, is nature's inexorable imperative.” – H. G. Wells

Lunchtime Meeting

October 23, 1998

QuOQuO


Outline

Background and motivation

Overview of QuO technology

The Toolkit project

Demonstration

QuOQuO


Large systems have become more distributed, yet many still have critical QoS requirements

CINCJ2

CINC OperationsPlanning Group

OPS/ INTELWORKSTATION

TARGET

CJTFPlanners

TARGET

MissionsCenters of GravityTask refinementForces refinementPhases refinementCOA evaluationCOA selection

FLTCINC MaritimePlanning Center

TMS CASES & HPC

Maritime & AirCampaign Assessment

JOINT FORCE MARITIMECOMPONENT CMDR

CASES TARGET

Task refinementIntegrated Target PrioritiesForces refinementSchedule refinementCOA evaluation

JOINT FORCE AIRCOMPONENT CMDR

TARGET ACPT

IDB

MissionsCOAsStrategy

Crisisassessment

COA evalCOA eval AirMar

XIDB

Target Nomination ListMaster Attack PlanAttack Plan Status

JFACCCombat Ops

Target Nomination ListWeaponeering

XIDB

APS/FLEX

TAMPS/COMPASS

JMCISIDB

Air Tasking Order (ATO)

NATOOPS/INTELWORKSTATION

Rear-echelonSim Center

toSHAPE

MasterTargetList

APS/FLEX RAAP

Refinement& eval

collaborative development

rehearse

Refinement& eval

ATO

Target data / Weaponeering


Distributed object middleware has emerged to solve heterogeneity and distribution problems

Middleware makes programming distributed applications easier

• Standard programming interfaces hide platform and system dependencies

• Standard protocols, e.g., message formats, allow applications on different systems to interoperate

• Middleware provides higher level, application oriented programming building blocks

Host 2

Impl

Host 1

Impl

Applications

Host 2

Simulation

Distributed ObjectMiddleware

Impl

IDLIDL

CollaborativePlanning

IDL

Hosts/Systems

Workflow


Wide-area distributed applications are still hard to build and maintain

Current DOC middleware is not sufficiently transparent with respect to real-time, fault tolerance, and other non-functional issues, and does not sufficiently accommodate adaptive behavior

• WANs are unpredictable, dynamic environments

• The configurations for an application changes over time

• Performance and system properties are buried under IDL’s functional interface so one can’t easily build an application that adapts to its changing environment and reuse code for a new environment

• Programmers end up programming around the DOC middleware to achieve real-time performance, predictability, security, etc.

ApplicationsSimulation


CollaborativePlanning

Hosts/Systems

Workflow

IDL IDL IDL


Distributed object middleware with QoS extensions is a powerful abstraction on which to build applications

Allows applications to specify both their functional requirements (IDL) and• QoS requirements and desires

• strategies for controlling and measuring QoS

• adaptation to react to changing QoS

Opens up the implementation• provides interfaces for QoS specification, measurement, and control

• supports application- and system-level adaptation

Applications


Hosts/Systems

IDL IDLIDL

SimulationCollaborativePlanning

Workflow

QuOQuO QuO


Outline



The Toolkit project

Demonstration

QuOQuO


The Quality Objects (QuO) framework supports development of distributed applications with QoS

QuOQuO

The QuO framework provides• Separation of concerns between software functional

properties and QoS needs

• Standard middleware interfaces between application and QoS-provider layers

• Facilities to enable application- and system-level adaptation

• Consistent interfaces for QoS measurement and resource management

Currently, QuO is being developed and used by four projects:• AQuA uses the QuO framework to manage dependability

• DIRM uses the QuO framework to manage network bandwidth

• OIT uses the QuO framework to manage survivability

• QuOIn uses QuO to integrate the properties of real-time, availability, managed communication, and security


System condition objects monitor QoS in the system

• system condition objects recognize changes in the system and notify the contracts that observe them

• QuO contracts notify client programs, users, managers, and other system condition objects through transition behavior

SystemCondition Objects

QuO applications specify, control, monitor, and adapt to QoS in the system

Application

Alternate Implementations

Contract (operating regions)

Servers

Network

ORB

Replication Mgr

Resource ReservationManager

IDS

Specification of operating regions, alternate implementations, and adaptation strategies using QuO’s QDL

Multiple layers of adaptation• managers and mechanisms can

adapt to changes in the system• QuO contracts provide another

layer of adaptation• Client and user can also adapt

Mechanisms and managers control QoS in the system

• a layer below QuO that provides ORB-level services, such as managed communi-cation, replication, or security

• contracts and delegates interface to these services through system condition objects

Figures copyright 1997, 1998 Xerox Corporation10 10/23/98 Lunchtime Meeting BBN Technologies

Simple QuO example application• Client displays images retrieved from a remote CORBA image server object

– Unreliability of remote server and contention for bandwidth with other applications makes performance and reliability unpredictable

– Capabilities built within the QuO framework improves performance and reliability

– Quorum’s DIRM project uses QuO to control resource reservation, assuring bandwidth

– Quorum’s AQuA project uses QuO to manage replication of the image server, improving reliability

Java Applet Client

Contract

SystemCondition

SystemCondition

CORBA/IIOP

ImageStore

QuO Kernel

ImageServer


QuO adds QoS control and measurement into the DOC remote method call

Client Network Server

ApplicationDeveloper

Qosketeer

MechanismDeveloper

Logical Method Call Client

Delegate

ORB Proxy

Specialized ORB

ContractSysCond

SysCond

SysCond SysCond

Object

Delegate

ORB Proxy

Specialized ORB

Contract

Network

Mechanism/PropertyManager

SysCond

SysCond

SysCond


A QuO application contains additional components (from traditional DOC applications)

• Contracts summarize the possible states of QoS in the system and behavior to trigger when QoS changes– Regions can be nested, representing different epochs at which QoS information becomes available, e.g., negotiated regions represent the levels of service a client expects to receive and a server expects to provide, while

reality regions represent observed levels of service

– Regions are defined by predicates over system condition objects

– Transitions specify behavior to trigger when the active regions change

• System condition objects are used to measure and control QoS– Provide interfaces to system resources, client and object expectations, mechanisms, managers, and specialized ORB functions

– Changes in system condition objects observed by contracts can cause region transitions

– Methods on system condition objects can be used to access QoS controls provided by resources, mechanisms, managers, and ORBs

• Delegates provide local state for remote objects– Upon method call/return, delegate can check the current contract state and choose behavior based upon the current state of QoS

– For example, delegate can choose between alternate methods, alternate remote object bindings, perform local processing of data, or simply pass the method call or return through


Measured capacity >= 10

As_expected:

Insufficient_resources:Measured capacity < 10

Contracts summarize system conditions into negotiated and reality regions and define transitions between them• Negotiated regions represent the expected behavior of client and server objects, and reality regions represent observed system behaviors

• Predicates using system condition objects determine which regions are valid

• Transitions occur when a region becomes invalid and another becomes valid

• Transitions might trigger adaptation by the client, object, ORB, or system

Normal:Expected capacity >= 10

Degraded:Expected capacity < 10Expected capacity >= 2

As_expected:

Extra_resources:

Measured capacity < 10Measured capacity >= 2

Measured capacity < 2Insufficient_resources:


Unusable:Expected capacity < 2

As_expected:

Extra_resources:

Measured capacity < 2


= Expected Region

= Reality Region


The QuO Toolkit provides tools for building QuO applications

• Quality Description Languages (QDL)– Support the specification of QoS contracts (CDL), delegates and their

adaptive behaviors (SDL), connection, creation, and initialization of QuO application components (TBD)

– QuO includes code generators that parse QDL descriptions and generates Java and C++ code for contracts, delegates, creation, and initialization

• QuO Runtime Kernel– Contract evaluator– Factory object which instantiates contract and system condition objects

• System Condition Objects, implemented as CORBA objects

CORBA IDL

CodeGenerators

CodeGenerators

Contract DescriptionLanguage (CDL)

QuO RuntimeQuO Runtime

Structure DescriptionLanguage (SDL)

Delegates Contracts


QuO GatewayQuO Gateway

IIOPGlue

Control

QuO gateways support specialized communication protocols and below the ORB mechanisms

Cli

ent-

Side

OR

B

IIOP Ensemble Group Comm. (AQuA)

WAN

RSVP resource res. (DIRM)

IIOP over TCP/IP (default)

IIOPGlue

Control

IIOP

Serv

er-S

ide

OR

B

• The QuO gateway enables insertion of below-the-ORB mechanisms and specialized network controls

• The gateway translates IIOP messages into specialized communication protocols or network level controls

• To the client-side, the QuO gateway looks like the remote ORB

• To the object-side, the QuO gateway looks like the client’s ORB

• The two ends of the gate-way are on the same LAN as the client/object

• Currently, we have gate-ways that support Ensemble group communication, RSVP resource reservation, and IIOP over TCP/IP


Architecture of a QuO application

SystemConditionObjects

QuO Runtime System(Java)

Application (C++ or Java)

FunctionalDelegate (C++ or Java)

premethod

postmethod

set client expectation

ORB set object expectation

system event

Contract


Client

Client CodeReference

Delegate

Proxy

Connect Callback

Factory

QuO Kernel

Syscond Syscond

ORB

Contract

Network Control Manager

QuO Kernel

Contract

SC SC

ORB

ORBProxyProxy

Object

1) Client calls delegate

2) Delegate evaluates contract

3) Measurement system conditions are signaled

4) Contract snapshots value of system conditions

5) Contract is re-evaluated

6) Region transitions trigger callbacks

7) Current region is returned

8) If QoS is acceptable, delegate passes the call to the remote object

9) Remote object returns value

10) Contract is re-evaluated...

11) Return value given to client

1

38

112

7

4

5

6

10

Del.

9

A sample operating sequence of a QuO application


Outline



The Toolkit project

Demonstration

QuOQuO


The Toolkit for Creating Adaptable Distributed Applications

• Funded under DARPA ITO’s Information Survivability, Survivability of Large Scale Systems program

• Two main goals– Develop QuO Toolkit technology

– Apply to and demonstrate in the area of survivability, e.g., intrusion detection, response, security

• Current status– One year into three year project

– Just delivered the first toolkit release, QuO v. 1.0

– Just starting work to apply QuO to survivability

QuOQuO


The QuO Toolkit provides tools for building QuO applications

• Quality Description Languages (QDL)– Analogous to CORBA’s Interface Description Language (IDL)

– Support the specification of QoS contracts

delegates and their adaptive behaviors

connection, creation, and initialization of QuO application components

– QuO includes code generators that parse QDL descriptions and generates Java and C++ code for contracts, delegates, creation, and initialization

• QuO Runtime Kernel– Contract evaluator

– Factory object which instantiates contract and system condition objects

• System Condition Objects– Implemented as CORBA objects

– We have a growing library of system condition objects for reuse


QuO’s Quality Description Languages (QDL)

• Contract Description Language (CDL) – expected regions of QoS

– reality regions of QoS

– transitions for adapting to changing levels of service

• Structure Description Language (SDL) – behavior alternatives for remote objects and their delegates

– alternate bindings and connection strategies

• Others, in progress

– Connection Description Language

– Resource Description Language

Implementation

CDL SDL ...

QDLIDL

QDL + IDLCompiler

QuOapplication

QuO Runtime

ORB


Quality Description Languages for specifying operating regions and adaptive behaviors

CORBA IDLtypedef sequence<long> LongSeq;interface Targeting { long calculate_distance_to_target(in long xcoord, in long ycoord); long identify_target(in long xcoord, in long ycoord);};

CodeGenerators

CodeGenerators

delegate behavior for Targeting and repl_contract is obj : bind Targeting with name SingleTargetingObject; group : bind Targeting with characteristics { Replicated = True };

call calculate_distance_to_target : region Available.Normal : pass to group; region Low_Cost.Normal : pass to obj; region Available.TooLow : throw AvailabilityDegraded; return calculate_distance_to_target : pass_through; default : pass_throughend delegate behavior; SDL

QuO RuntimeQuO Runtime

ContractDelegate

CDL

contract Replication( syscond ValueSC ValueSCImpl ExpectedReplicas, callback AvailCB ClientCallback, syscond ValueSC ValueSCImpl Measured, syscond ReplSC ReplSCImpl ReplMgr ) isnegotiated regions are region Low_Cost : ... region Available : when ClientExpectedReplicas > 1 => reality regions are region Low : when Measured < ExpectedReplicas => region Normal : when Measured > ExpectedReplicas => transitions are transition any->Low : ClientCallback.availability_degraded(); ... transitions are ...end Replication;


CDL contract to control object replicationcontract Replication( syscond ValueSC ValueSCImpl ClientExpectedReplicas, callback AvailCB ClientCallback, syscond ValueSC ValueSCImpl MeasuredNumberReplicas, syscond ReplSC ReplSCImpl ReplMgr ) isnegotiated regions are region Low_Cost : when ClientExpectedReplicas == 1 => reality regions are region Low : when MeasuredNumberReplicas < ClientExpectedReplicas => region Normal : when MeasuredNumberReplicas == ClientExpectedReplicas => region High : when MeasuredNumberReplicas > ClientExpectedReplicas => transitions are transition any->Low : ClientCallback.availability_degraded(); transition any->Normal : ClientCallback.availability_back_to_normal(); transition any->High : ClientCallback.resources_being_wasted(); end transitions; end reality regions; region Available : when ClientExpectedReplicas > 1 => reality regions are region Low : when MeasuredNumberReplicas < ClientExpectedReplicas => region Normal : when MeasuredNumberReplicas > ClientExpectedReplicas => transitions are transition any->Low : ClientCallback.availability_degraded(); transition any->Normal : ClientCallback.availability_back_to_normal(); end transitions; end reality regions; transitions are transition Low_Cost->Available : ReplMgr.adjust_degree_of_replication(ClientExpectedReplicas); transition Available->Low_Cost : ReplMgr.adjust_degree_of_replication(ClientExpectedReplicas); end transitions; end negotiated regions;end Replication;


SDL code that supports choosing between replicated and non-replicated server objects

delegate behavior for Targeting and Replication is

call calculate_distance_to_target :

region Available.Normal :

pass to calculate_distance_to_target_multicast;

region Low_Cost.Normal :

pass to calculate_distance_to_target_multicast;

region Available.Low :

java_code { System.out.println(“Remote call would fail”);

retval = -1; };

cplusplus_code { cerr << “Remote call would fail”);

retval = -1; };

return calculate_distance_to_target :

pass_through;

default : pass_through

end delegate behavior;

• SDL supports choosing between methods, run-time binding, and embedded Java or C++ code.


Motivation for applying QuO to survivability

• Large scale information systems are vulnerable to attack

• Most large scale systems rely on a single implementation

• Distributed object systems and wide-area networks offer increased chances of failure or attack

• If applications had the ability to detect abnormal behavior indicating failures, intrusions, or attacks and adapt to avoid them, the applications would be more likely to survive hostile situations

• Current distributed object systems do not provide the mechanisms and infrastructure necessary to support this


Open Implementation Toolkit - Enabling Technologies for Building Adaptable, Survivable Systems

• Allow objects, subsystems, and applications to specify alternate implementations, service requirements, constraints, and normal operating behavior of each implementation

• Provide mechanisms for monitoring runtime behavior and system characteristics

• Provide mechanisms for recognizing when implementations are operating outside their acceptable ranges, which might indicate intrusions, attacks, or failures

• Support notifying system and application components of the anomalous behavior, dynamically selecting alternate behavior, and reconfiguring to avoid problem areas

• Demonstrate and validate the toolkit in cooperation with other participants in the DARPA/ITO Survivability of Large Scale Systems program

• Integrate results into the unified QuO framework

Adaptive Application

Alternate Implementations

CORBA + QuO

Server

Server ServerCorruptedServer

Broken Connection


QuO Toolkit support for intrusion detection

• Encode normal behavior as regions in QuO contracts– applications operating outside the normal regions indicate potential intrusions

• Encode known attack patterns as regions in QuO contracts– applications operating inside known attack regions indicate potential intrusions

• Interface to existing and emerging intrusion detection systems (IDSs)– QuO provides a means (system condition objects) in which IDSs can interoperate among themselves and with other property managers

– QuO’s system condition objects provide a single application level interface to all the different IDS interfaces

– QuO supports the building of applications that run in different survivability modes, from paranoid to intrusion unaware, and can switch among these at runtime


QuO Toolkit survivability partners

• University of Illinois– UIUC has developed a dependability manager, Proteus

– Timing and value faults recognized by Proteus are potential intrusions

– In addition to fault recovery, Proteus collects information and notifies the QuO/application layer

• Odyssey Research Associates– Performing research in computer immunology

– Identify patterns of normal usage and recognize when a system is operating outside normal regions

• MIT– Performing research in intrusion detection and building IDSs


Where to find more information

• QuO

http://www.dist-systems.bbn.com/tech/QuO

• The Toolkit project

http://www.dist-systems.bbn.com/projects/OIT

• Toolkit personnel

http://www.dist-systems.bbn.com/people

• To get the QuO Toolkit v. 1.0, send mail to

[email protected]

[email protected]


Outline



The Toolkit project

Demonstration

QuOQuO

Date post:	03-Jan-2016
Category:	Documents
Upload:	kristin-atkinson
View:	212 times
Download:	0 times

1 10/23/98Lunchtime Meeting BBN Technologies Toolkit for Creating Adaptable Distributed Applications...

Documents