Post on 18-Dec-2014
description
transcript
www.s-cube-network.eu
S-Cube Learning Package
Run-time Verification for Preventive Adaptation
University of Duisburg-Essen (UniDue), Center for
Scientific and Technological Research (FBK), Politecnico
di Milano (POLIMI)
Eric Schmieders, UniDue
© S-Cube
Learning Package Categorization
S-Cube
Quality Definition, Negotiation
and Assurance
Quality Assurance and Quality Prediction
Run-time Verification
for Preventive Adaptation
Learning Package Overview
Problem Description
Preventive Adaptation based on Runtime-Verification
Discussion
Conclusions
© S-Cube
Service-oriented Systems Need for Adaptation
Highly dynamic changes due to
– 3rd party services, multitude of service providers, …
– evolution of requirements, user types, …
– change in end-user devices, network connectivity, …
Difference from traditional
software systems
– Unprecedented level of change
– No guarantee that 3rd party
service fulfils its contract (SLA)
– Hard to assess behaviour
of infrastructure at design time
© S-Cube
Let’s Consider a Scenario (1)
Assume you are a provider of a composite service
Commercial customers order products via a Web service
interface
Your process checks an internal stock, orders some missing
parts, and assembles the product
© S-Cube
Let’s Consider a Scenario (2)
(Of course) there are contractual obligations
– Delivery in time
– Availability of your service
– Product quality
These can be formulated using Service Level Agreements
© S-Cube
– Note: in this context not necessarily
WSLA style machine processable
SLAs – can also be just regular
contracts
Let’s Consider a Scenario (3)
As a provider, you want to receive timely notifications if these
obligations are likely to be violated
– Usually there are penalties to be paid if contractual obligations are
(repeatedly) violated
– Even if this is not the case, customer satisfaction will suffer and
existing customers may terminate their contracts
© S-Cube
Based on this notifications …
– … countermeasures can be taken (i.e.,
adaptations can be triggered)
– … customers can be notified
Learning Package Overview
Problem Description
Preventive Adaptation based on Runtime-Verification
Discussion
Conclusions
© S-Cube
Reactive Adaptation
– Repair/compensate for external failure visible
to the end-user
Preventive Adaptation
– An internal failure/deviation occurs
Will it lead to an external failure?
– If “yes”: Repair/compensate internal
failure/deviation to prevent external failure
Proactive Adaptation
Is internal failure /deviation imminent
(but did not occur)?
– If “yes”: Modify system before internal
failure/deviation actually occurs
Types of Adaptation
Failure? Failure!
Failure?
Failure!
For Preventive Adaptation
– Data Mining: Extract knowledge from “historical” data - TUW & USTUTT: “Prediction and Prevention of SLA Violations in Composite Services”
- SZTAKI: “Historical Data Based Predictions for Resource Allocation”
–Run-time Verification: Formally ascertain that properties hold - Paluno, FBK, Polimi: “Assumption-based Run-time Verification”
For Proactive Adaptation
– Simulation: Execute dynamic models to simulate behavior - UPM & TUW: “Simulation of Provision Resources”
- INRIA: “QoS Assurance Using Discrete-Event Simulation”
– Static Analysis: Fomally infer properties (without execution) - UCBL & UPM: “Soft Constraints for QoS-Aware Service Selection”
- UPM: “Data-aware Resource Analysis for Service Orchestrations”
– Online Testing: Actively execute services in parallel to normal use - Paluno & UPC: “Augmenting Monitoring with Online Testing”
- SEERC & Paluno : “Just-in-time Testing of Conversational Services”
Types of Adaptation
[Deliverable JRA-1.3.5, http://www.s-cube-network.eu/qp/]
© S-Cube
Background: Runtime Verification
© S-Cube
Runtime Verification is a system analysis approach based on extracting
information from a running system extracted information is used to detect and
possibly react to observed behaviors satisfying or violating certain properties.
Properties can be functional (certain functionality) or non-functional (e.g. required
response time)
Runtime verification specifications are typically expressed in regular expressions,
context-free patterns, linear temporal logics, etc.
Based on formal requirements specifications, monitors are created and applied
during the systems runtime
Runtime verification can be used for many purposes: security or safety policy
monitoring, debugging, testing, verification, validation, profiling, fault protection,
etc…
[BianculliEtAl2008, GehlertEtAl2010, JRA-1.3.2,
http://en.wikipedia.org/wiki/Runtime_verification]
SPADE An Example for Runtime Verification
© S-Cube
Background:
S-Cube Service Life-Cycle
A life cycle model is a process model that covers the activities related to
the entire life cycle of a service, a service-based application, or a software
component or system. [Gu2007, http://www.s-cube-
network.eu/km/terms/l/life-cycle-model]
The SPADE approach uses runtime verification for preventive adaptation
ON THE NEXT SLIDES: SPADE will be used to illustrate preventive
adaptation by applying run-time verification
Furthermore
– (A) an abstract example scenario and
– (B) the S-Cube Service Life-Cycle
will be used to illustrate the design time and run time activitives of
SPADE
Runtime Verification with SPADE
SPADE (Specification and Assumption based Detection of Adaptation
Needs) equips SBAs with adaptation capabilities, empowering them to
adapt preventively [MetzgerEtAl2010, SchmiedersEtAl2011]
TO ACHIEVE THIS: SPADE uses
– run-time verification techniques
– execution data of the monitored instances
– assumptions concerning the SBAs' context, e.g. the response time of
constituent services (derived from SLAs: http://www.s-cube-
network.eu/km/terms/s/service-level-agreement)
– formalized requirements of the SBA
This data is used for performance prediction and preventive adaptation
which is able to avoid requirement violations of running SBAs
© S-Cube
Action 5
:FastWeather
:CurrencyConverter
:HyperlinkExtractor
Action 7
:WSAmazonBox
Action 2 :Google
Action 1
Action 4
:GetJoke Action 6
Action 3
5900 ms
8400 ms
6400 ms
14000 ms
2700 ms
4000 ms
Abstract Example
© S-Cube
Action 5
:FastWeather
:CurrencyConverter
:HyperlinkExtractor
Action 7
:WSAmazonBox
Action 2 :Google
Action 1
Action 4
:GetJoke Action 6
Action 3
5900 ms
8400 ms
6400 ms
14000 ms
2700 ms
4000 ms
Abstract Example
Activity
Diagram
of Workflow
3rd Party
Service
Assumed
response time
Service Binding
© S-Cube
S-Cube Service Life-Cycle
Evolution
Deployment &
Provisioning
Identify
Adaptation
Need
Identify
Adaptation
Strategy
Enact
Adaptation
Adaptation
Operation &
Management
Design
Requirements
Engineering
Realization
Requirements
R := {r1, r2 …}
Workflow
Specification S,
Service
Candidates C
SLA Negotiation
A:= {a1, a2, …} Deployment
Check: S, A |= R
Monitoring
m1 |= a1
(vgl. SM)
Check: S, A‘, M |= R
[MetzgerEtAl2010, SchmiedersEtAl2011]
Solve:
C, S |= R
Create:
Plan p Enact p
© S-Cube
Requirements Engineering
In the requirements engineering phase, the functional
and quality requirements for the SBA are elicited and
documented!
GOAL: enable check, whether application
will from its requirements during operation?
functional and non-functional requirements
are formally expressed as input to SPADE
SPADE uses the specification language ALBERT to formalize the SBA
requirements
© S-Cube
Requirements
Engineering
Requirements
R := {r1, r2 …}
For more information on ALBERT:
Bianculli, D., Ghezzi, C., Spoletini, P., Baresi, L., Guinea, S.: A guided tour through
SAVVY-WS: A methodology for specifying and validating web service compositions.
In: Borger, E., Cisternino, A. (eds.) Advances in Software Engineering. LNCS, vol.
5316. Springer (2008)
Requirements Engineering
Example:
– Formalize Requirement r: end-to-end response time is at most 55
seconds
– ALBERT is used to formalize r:
r := onEvent(start, "Action1") -> Within( onEvent( end,
"Action7"), 55000)
– onEvent operator evaluates to true if the activity specified in its
second argument performs the state change denoted in its first
argument.
– Within operator evaluates to true if its first argument evaluates to true
within the amount of milliseconds specified in its second argument
© S-Cube
Design
During the design phase, the activities and
the control flow of the application are
specified!
Definition of workflow in e.g. BPEL
Candidate services are identified that can
provide the required functionality/quality
SPADE extends idea of Bianculli et al. [BianculliEtAl2008] by using a
model checker during run-time
ALBERT expressions can be executed by BOGOR: formalize the workow
using BIR, the BOGOR Input Representation.
© S-Cube
Design
Workflow
Specification S,
Service
Candidates C
Design
Example:
BIR file (excerpt) as input for the BOGOR Model Checker:
© S-Cube
…
Background: Model Checking
© S-Cube
With model checking it is automatically testable, whether a model meets a given
specification.
Concept is general and applies to all kinds of logics and suitable structures
Example: verifying whether a given formula in the propositional logic is satisfied by
a given structure
Typically hardware or software systems are checked
The checked specification contains safety requirements, e.g. the absence of
deadlocks, critical states that can cause the system to crash, …
The model of the system and the specification are formulated in precise
mathematical language (e.g. LTL, CTL…)
Common model checker: http://nusmv.fbk.eu/, http://spinroot.com/spin/whatispin.html,
http://www.uppaal.org/, and http://babelfish.arc.nasa.gov/trac/jpf
[GehlertEtAl2011, Bianculli2008, http://de.wikipedia.org/wiki/Model_Checking]
Realization
To achieve the desired end-to-end quality of
SBAs, contracts between service providers
and service consumers on quality aspects
of services are established!
For each candidate service: negotiate best quality
level for the available budget & stipulated in an SLA
SPADE: the quality levels are formalized, that have been negotiated and
agreed upon with the service providers
Quality levels are used as assumptions A about the SBA's context
ALBERT is used to formalize A
© S-Cube
Realization
SLA Negotiation
A:= {a1, a2, …}
For more information on negotiating SLAs:
Comuzzi, M., Pernici, B.: A framework for qos-based web service contracting. ACM
Transactions on web 3(3) (2009)
Realization
Example:
– For our example we use the assumed response time given in the
example SBA.
– The assumption for the aFastWeather service bound to Action 1, is formalized
as follows1:
aFastWeather := onEvent (start, "Action 1")
Within( onEvent(end, "Action 1" ), 5900)
© S-Cube
Background:
Assumptions
In the case of service-based applications, assumptions may characterize the
constituent services (e. g., their interfaces, QoS parameters, etc.) and/or the
context (e. g., infrastructure, business, context, user, etc.).
A violation of those assumptions may lead to a situation, in which the software
system does not provide the expected quality anymore and, therefore, deviates
from its requirements.
[GehlertEtAl2010]
Deployment
The deployment and provisioning phase
comprises all the activities needed to make
the SBA available to its users!
SPADE uses BOGOR to check whether the
workflow specification (S), under the given
assumptions (A), satisfies the requirements (R),
i.e. whether:
S, A |= R
If requirements are not satisfied phases of the evolution loop are re-
executed, e.g., in order to bind faster services.
Example:
– The specification of the abstract service composition evaluates to true, i.e. S
and A satisfy R. Thus, the SBA is deployed.
© S-Cube
Deployment &
Provisioning
Deployment
Check: S, A |= R
Operation and Management
This phase comprises the execution of
the SBA and the monitoring of its
constituent services using service
monitoring techniques!
To identify assumption violations
during the operation of the SBA monitoring
mechanisms are employed
SPADE uses the monitoring facilities of the runtime environment to
CHECK: m |= a
Example:
– Assume: mFastWeather and mGoogle satisfy their related assumptions
– BUT: WSAmazonBox too late mGoogle |≠ aGoogle.
– CONSEQUENCE: Due to the deviation the next phase is entered
immediately; a performance violation could be indicated!
© S-Cube
Operation &
Management
Monitoring
m1 |= a1
(vgl. SM)
Identify Adaptation Needs
In this phase the need for the adaptation
is detected. To this end a runtime
verification against the SBAs requirements
is performed!
SPADE checks: are requirements still satisfied?
For all invoked services:
– SPADE replaces the assumption (A) values by concrete monitoring data (M)
– only for the not invoked services, assumptions (A’) are used
SPADE thus uses a subset A’ (of A), together with the set of monitored
data M to check whether: S,M,A’ |= R
SPADE utilizes BOGOR to perform this verification during run-time
If there is one future path which violates r adaptation must be
triggered
ELSE the workflow execution is continued
© S-Cube
Identify
Adaptation
Need
Check: S, A‘, M |= R
Identify Adaptation Needs
Example:
– Check whether the requirement is still met (S,M,A’ |= R):
- workflow specification S
- the monitoring data (mFastWeather, mGoogle and mWSAmazonBox)
- assumptions of the to be invoked services (aHylinkExtractor, aCurrencyConverter,
aGetJoke)
checked against the requirement r.
– Predicted end-to-end duration: 56238 ms exceeds the 55 seconds
demanded by r.
– CONSEQUENCE: the workflow has to be adapted, otherwise a
performance violation seems to be in all probability
© S-Cube
Action 5
:FastWeather
:CurrencyConverter
:HyperlinkExtractor
Action 7
:WSAmazonBox
Action 2 :Google
Action 1
Action 4
5735 ms
:GetJoke Action 6
Action 3
5900 ms
8400 ms
6400 ms
14000 ms
2700 ms
4320 ms
4000 ms
15083 ms
Identify Adaptation Needs
Re
sp
on
se
Tim
es
6400 ms
14000ms
2700 ms
Action 5
:FastWeather
:CurrencyConverter
:HyperlinkExtractor
Action 7
:WSAmazonBox
Action 2 :Google
Action 1
Action 4
5735 ms
:GetJoke Action 6
Action 3
5900 ms
8400 ms
6400 ms
14000 ms
2700 ms
4320 ms
4000 ms
15083 ms
FAILURE
Identify Adaptation Needs
Re
sp
on
se
Tim
es
A
ss
um
pti
on
s
∑ > e2e_requirement(55 ms)
e2e is predicted to be violated!
Adaptation necessary!
Identify Adaptation Strategy
In this phase an adaptation strategy is created.
The strategy is supposed to avert the menacing
SLA violation successfully when enacted!
SPADE is equipped with service substitution
capabilities
SPADE exploits the CHOCO constraint solver
to determine which not-yet-invoked services
have to be substituted (http://www.emn.fr/z-info/choco-solver/)
Find a strategy with CHOCO for every future violating path (necessary, as
it’s unclear which path will be taken in the further execution)
Example:
– Two paths are violating, i.e. p1 = (Action5, Action7) and p2 = (Action5,
Action6, Action7) paths will violate r and therefore have to be adapted.
– Based on the output of CHOCO: services for Action5 and Action6 are chosen
to be substituted by faster services
© S-Cube
Identify
Adaptation
Strategy
Solve:
C, S |= R
and create
Plan p
Enact Adaptation
During this last adaptation phase the adaptation
strategy is executed. For this purpose, the
instructions comprised in the adaptation strategy
are dispatched!
Dispatching an adaptation strategy usually utilizes
– the facilities provided by the chosen run-time environment
– involves additional adaptation mechanisms
SPADE:
– uses the interception-mechanisms provided by the runtime
environment to reroute messages to the new target service
– switchs the target service of the invocation to the service identified
during the previous phase.
© S-Cube
Enact
Adaptation
Enact p
Enact Adaptation
Example:
– In our example, the two service invocations Action5 and Action6 are
redirected to the substituting services.
– the message routing table comprising the message destinations is
manipulated.
Consequently, the execution of the instance is resumed..
© S-Cube
Learning Package Overview
Problem Description
Preventive Adaptation based on Runtime-Verification
Discussion
Conclusions
© S-Cube
Is Run-time Verification effective when adapting preventively? SPADE can be considered as one representative for Run-time
Verification techniques
We conducted an experiment to access the effectiveness of
SPADE
Our experimental measurement of SPADE's effectiveness is
twofold
– false positives: unnecessary adaptations are measured
– Impossible adaptations: amount of situations in which SPADE is
unable to perform an adaptation is counted
© S-Cube
Is Run-time Verification useful for adapting preventively? Determining the Degree of False Positives
1. Calculated Response Time:
– Calculate the end-to-end response time for each SBA instance execution
– Add the monitored response times of the invoked services along the SBA
instance's path
Based on this determine false positive adaptation triggers:
2. SPADE triggered an adaptation
3. Check: calculated SBA instance response time violates the SBAs end-to-end
requirements?
False Positive, if: 1, 2, and 3 reveal that the requirement would not have
been violated in case the SBA execution continues without adaptation
© S-Cube
Is Run-time Verification useful for adapting preventively? Determining the Degree of Impossible Adaptations
Situations during experimentation were observed:
– service invocations deviated from their stipulated response time, such that the
performance requirement is violated
– too late to apply SPADE as the performance requirement is already violated
determine the percentage of these situations:
relate the number of the service invocations which lead to these SLA
violations to the amount of all service invocations
© S-Cube
Is Run-time Verification useful for adapting preventively? Experimental Results:
SPADE has been applied to 5884 SBA instances (cf. row (a) in Table below).
629 of those SBA instances have been executed without any assumption violations
(b), in 5255 SBA instances assumptions have been violated (c), for those
instances, SPADE has identified 604 preventive adaptation triggers (d)
72 of the adaptation triggers were false positives (e) just 1.2% of the workflow
instances would have been unnecessarily adapted.
With respect to the challenging time constraint of 55 seconds, the very low
percentage of false positives can be considered as extremely promising
© S-Cube
Is Run-time Verification useful for adapting preventively? Amount of situations in which SPADE cannot adapt is very low as well
Row (c) a total of 825 out of 28624 service invocations, i.e. 2.5%, lead to
situations where an adaptation is not possible.
BUT: this could still mean a threat for applications which are required to
meet the requirements in nearly 100%, e.g. in an emergency scenario
© S-Cube
Learning Package Overview
Problem Description
Preventive Adaptation based on Runtime-Verification
Discussion
Conclusions
© S-Cube
Conclusion
SPADE is an automated technique to determine adaptation
needs to trigger preventive adaptations of SBAs
To achieve this high degree of automation SPADE exploits
runtime verification techniques
The experiments reveal situations in which SPADE cannot
perform a preventive adaptation, that again: SPADE was able
to avoid SLA violations by a high percentage
This indicates the need for further improvement of
SPADE, but also generally demonstrates the effectiveness
of applying runtime verification techniques for preventive
adaptation
© S-Cube
References
© S-Cube
[MetzgerEtAl2010] Metzger, A.; Schmieders, E.; Cappiello, C.; Di Nitto, E.; Kazhamiakin, R.; Pernici, B. & Pistore, M.: Towards Proactive Adaptation: A Journey along the S-Cube Service Life-Cycle. MESOA: 4th International Workshop on Maintenance and Evolution of Service-Oriented Systems, 2010
[SchmiedersEtAl2011] Schmieders, E. & Metzger, A.: Preventing Performance Violations of Service Compositions using Assumption-based Run-time Verification. ServiceWave, 2011
[Gu2007] Q. Gu and P. Lago, A stakeholder-driven service life cycle model for SOA. In 2nd international Workshop on Service Oriented Software Engineering: in Conjunction with the 6th ESEC/FSE Joint Meeting (Dubrovnik, Croatia, September 03 - 03, 2007). IW-SOSWE '07. ACM, New York, NY, 1-7.
[BianculliEtAl2008] Bianculli, D.; Ghezzi, C.; Spoletini, P.; Baresi, L. & Guinea, S.: A Guided Tour through SAVVY-WS: A Methodology for Specifying and Validating Web Service Compositions, Springer-Verlag, 2008, 131-160
[GehlertEtAl2010] Gehlert, A.; Bucchiarone, A.; Kazhamiakin, R.; Metzger, A.; Pistore, M. & Pohl, K.: Exploiting assumption-based verification for the adaptation of service-based applications. SAC '10: Proceedings of the 2010 ACM Symposium on Applied Computing, ACM, 2010, 2430-2437
[ComuzziEtAl2009] Comuzzi, M., Pernici, B.: A framework for qos-based web service contracting. ACM Transactions on web 3(3) (2009)
[JRA-1.3.2] http://www.s-cube-network.eu/working-area/activities-and-workpackages/jra-1/WP-JRA-1.3/deliverables/cd-jra-1.3.2/CD-JRA-1.3.2_Quality%20Reference%20Model%20for%20SBA.pdf/view
Further S-Cube Reading
© S-Cube
Leitner, Wetzstein, Rosenberg, Michlmayr, Dustdar, and Leymann. Runtime Prediction of Service Level Agreement Violations for Composite Services. In Proceedings of the 2009 International conference on Service-Oriented Computing (ICSOC/ServiceWave'09), Springer-Verlag, Berlin, Heidelberg, 176-186.
Leitner, Michlmayr, Rosenberg, and Dustdar. Monitoring, Prediction and Prevention of SLA Violations in Composite Services. In Proceedings of the 2010 IEEE International Conference on Web Services (ICWS '10). IEEE Computer Society, Washington, DC, USA, 369-376.
Wetzstein, Leitner, Rosenberg, Brandic, Dustdar, and Leymann. Monitoring and Analyzing Influential Factors of Business Process Performance. In Proceedings of the 13th IEEE international conference on Enterprise Distributed Object Computing (EDOC'09). IEEE Press, Piscataway, NJ, USA, 118-127.
Acknowledgements
The research leading to these results has
received funding from the European
Community’s Seventh Framework
Programme [FP7/2007-2013] under grant
agreement 215483 (S-Cube).
© S-Cube