Generation of Conformance Generation of Conformance Test Suites for Compositions Test Suites for Compositions of Web Services Using Model of Web Services Using Model

CheckingCheckingJosé García-FanjulJosé García-Fanjul, Claudio de la Riva and Javier Tuya, Claudio de la Riva and Javier Tuya

University of Oviedo (Spain)University of Oviedo (Spain)

This work is supported by the Ministry of Science and Education (Spain) under This work is supported by the Ministry of Science and Education (Spain) under the National Program for Research, Development and Innovation, projects the National Program for Research, Development and Innovation, projects

IN2TEST (TIN2004-06689-C03-02) and REPRIS (TIN2005-24792-E).IN2TEST (TIN2004-06689-C03-02) and REPRIS (TIN2005-24792-E).

TAIC PART. Testing: Academic & Industrial Conference.TAIC PART. Testing: Academic & Industrial Conference.

Windsor (UK) - August, 2006Windsor (UK) - August, 2006

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 22))

MotivationMotivation

Investment in web services software Investment in web services software is increasing [IDC, 2006]:is increasing [IDC, 2006]:• Doubling from 2003 to 2004 (reaching Doubling from 2003 to 2004 (reaching

$2,3 billion).$2,3 billion).• (Expectedly) becoming $15 billion by (Expectedly) becoming $15 billion by

2009.2009. … … but there are not many research but there are not many research

works on testing web services works on testing web services software.software.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 33))

Do we need new testing methods Do we need new testing methods for web services?for web services?

Testing web services software is different.Testing web services software is different. Some unresolved challenges (from [Zhang Some unresolved challenges (from [Zhang

and Zhang, 2005] and [Canfora and Di and Zhang, 2005] and [Canfora and Di Penta, 2006])Penta, 2006])• The need to remotely test web services, with The need to remotely test web services, with

its associated cost.its associated cost.• Lack of observability of the service code and Lack of observability of the service code and

structurestructure..• The ability to dynamically search and invoke The ability to dynamically search and invoke

web services.web services.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 44))

The research hypothesisThe research hypothesis

A new method for generating test suites A new method for generating test suites for compositions of web services is for compositions of web services is needed, with these characteristics:needed, with these characteristics:• It will be static (no need to execute the It will be static (no need to execute the

software for obtaining the test cases).software for obtaining the test cases).• The selection of the test cases will be guided The selection of the test cases will be guided

by adequacy criteria.by adequacy criteria.• The only required input will be a specification The only required input will be a specification

of the composition (BPEL).of the composition (BPEL). No knowledge about the partners particular No knowledge about the partners particular

behaviour.behaviour.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 55))

Background: Model checkingBackground: Model checking

Model checker

Model Property

Counterexample

A formal verification technique to automatically A formal verification technique to automatically ascertain if a property holds in a model.ascertain if a property holds in a model.

1) A model must bebuilt for the systemwe want to check.

2) Properties mustbe specified.

3) The tool (model checker)searches all the possible states within the model.

4) If the property does not hold, it provides a counterexample showing its violation.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 66))

Using model checking for Using model checking for software testing software testing [Ammann, 1998][Ammann, 1998]

Model checker

Model Property

Counterexample

To obtain a test case for a certain condition C.To obtain a test case for a certain condition C.

Model of thesoftware

C NEVERholds

Counterexample (fulfilling C)

1) The model checker(for instance, SPIN)is fed with a model for the software…

2) …and a LTL formula stating that C never holds.

3) The output obtainedfrom the tool is a counterexample in which the software fulfils C.

4) That counterexample can be transformed into a test case, as it describes an execution of the software in which the desired test condition holds.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 77))

Overview of the methodOverview of the methodBPEL

Transforming BPEL to

PROMELA

Applying adequacy

criteria

Test case specification

Model checker

Model Property

Counterexample

1) A model will bebuilt from the BPELspecification of thebusiness process.

2) Adequacy criteria will be used to select test requirements. So:•The PROMELA code will be instrumented to discern if an execution meets a requirement.•LTL properties will be properly constructed (the negation of the requirements).

3) The model checkeris executed, andcounterexamplesobtained.

4) Counterexamples are transformed into test cases specifications.

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 88))

Preliminary workPreliminary work

The method has been applied to a The method has been applied to a sample composition called “loan sample composition called “loan approval”.approval”.• Its objective is to conclude whether a Its objective is to conclude whether a

certain request for a loan will be certain request for a loan will be approved or not.approved or not.

The chosen criterion has been a The chosen criterion has been a transition coverage criterion.transition coverage criterion.

A representation of the “loan approval” sample composition.A representation of the “loan approval” sample composition.

1) Receives a requestfrom a partner called “customer”

2) The “assessor”Partner measures therisk associated with low amount requests.

3) Requests made for a large amount of money or which are evaluated by the assessor as not having a low risk willbe examined by another partner (“approver”).

Test cases for the “loan approval” sample composition.Test cases for the “loan approval” sample composition.

First test case: Transition #1LTL property: [] (!tran1)

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 1111))

0 :in run customer() 0 :in run customer()

1 cus request.firstName 1 cus request.firstName

Proce Statement customer customer customer customer Proce Statement customer customer customer customer

1 cus request.name = und 0 0 3 0 1 cus request.name = und 0 0 3 0

1 cus request.amount = 3 0 0 3 3 1 cus request.amount = 3 0 0 3 3

0 :in run bpel_loan_appr 0 3 3 3 0 :in run bpel_loan_appr 0 3 3 3

2 bpe f1a1==0 0 3 3 3 2 bpe f1a1==0 0 3 3 3

Proce Statement bpel_loa bpel_loa bpel_loa bpel_loa bpel_loa customer Proce Statement bpel_loa bpel_loa bpel_loa bpel_loa bpel_loa customer customer customer customer customer customer customer

0 :in run approver() 0 0 0 0 0 0 3 0 :in run approver() 0 0 0 0 0 0 3 3 3 3 3

0 :in run assessor() 0 0 0 0 0 0 3 0 :in run assessor() 0 0 0 0 0 0 3 3 3 3 3

1 cus values: 1!3,3,3 0 0 0 0 0 0 3 1 cus values: 1!3,3,3 0 0 0 0 0 0 3 3 3 3 3

1 cus BPEL_loanApprovalP 0 0 0 0 0 0 3 1 cus BPEL_loanApprovalP 0 0 0 0 0 0 3 3 3 3 3

Proce Statement BPEL_loa bpel_loa bpel_loa bpel_loa bpel_loa bpel_loa Proce Statement BPEL_loa bpel_loa bpel_loa bpel_loa bpel_loa bpel_loa customer customer customer customer customer customer customer customer

2 bpe values: 1?3,3,3 [3,3,3] 0 0 0 0 0 0 2 bpe values: 1?3,3,3 [3,3,3] 0 0 0 0 0 0 3 3 3 3 3 3

[...][...]

Extracting a test case from the counterexampleExtracting a test case from the counterexample..

customer: request.amount = 3

customer: BPEL_loanApprovalPort_IN!request

bpel: request.amount<4

bpel: tran1 = true

bpel: loanassessor_riskAssessmentPort_IN!request

assessor: riskAssessment.risk = low

assessor: loanassessor_riskAssessmentPort_OUT! riskAssessment

bpel: tran3 = true

bpel: approvalInfo.accept = yes

bpel: tran5 = true

bpel: BPEL_loanApprovalPort_OUT!approvalInfo

bpel: bpel_ends = true

INPUTS

1. the customer makes a request for an amount of 3 (less than four)

2. the risk assessment from the assessor is low

EXPECTED OUTPUT

The reply to the customer is affirmative.

Test cases for the “loan approval” sample composition.Test cases for the “loan approval” sample composition.

First test case: It covers transition #1… but also #3 and #5.

Test cases for the “loan approval” sample composition.Test cases for the “loan approval” sample composition.

First test case: It covers transition #1… but also #3 and #5.

Second test case: It covers transitions 2 and 6.

Test cases for the “loan approval” sample composition.Test cases for the “loan approval” sample composition.First test case: It covers transition #1… but also #3 and #5.

Second test case: It covers transitions 2 and 6.

Third test case: It covers transition 4 (and also 1 and 6).

TAIC PART – Windsor (UK) - August, 2006TAIC PART – Windsor (UK) - August, 2006 José García-Fanjul (Page José García-Fanjul (Page 1515))

Expected contributionsExpected contributions Main contribution:Main contribution:

• Definition of a new method to obtain conformance test Definition of a new method to obtain conformance test suites for compositions of web services. suites for compositions of web services. The method will The method will rely on a model checking tool (SPIN) for obtaining the rely on a model checking tool (SPIN) for obtaining the test cases specifications.test cases specifications.

Specifically, research will address how to:Specifically, research will address how to:• Transform a BPEL specification to a PROMELA model.Transform a BPEL specification to a PROMELA model.• Instrument PROMELA code, considering the adequacy Instrument PROMELA code, considering the adequacy

criteria.criteria.• Construct LTL properties for the counterexamples to Construct LTL properties for the counterexamples to

show sample executions of the model that meet the show sample executions of the model that meet the criteria.criteria.

• Automatically obtain a test suite specification from the Automatically obtain a test suite specification from the counterexamples that SPIN provides.counterexamples that SPIN provides.

Thank you for your attentionThank you for your attention

José García-FanjulJosé García-Fanjul

jgfanjul@uniovi.esjgfanjul@uniovi.es