Using Interactive Genetic Algorithm for Requirements Prioritization

Using Interactive

Genetic Algorithm for Requirements Prioritization

Francis Palma

Department of Information Engineering and Computer Science Universitá degli Studi di Trento

[email protected]

December 3, 2010

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Presentation Outline

Using Interactive


Problem description

Related works

Genetic Algorithm

Our approach

Case study

Results

Conclusions

2

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Problem Description

Using Interactive


Problem: [Prioritization of Requirements] To find the best ordering of requirements in each successive release to ensure quality & value of the delivered system, trade-off constraints & end-user satisfaction. 3

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Do Trade-off between user needs & real constraints

Now, highest quality &

best valued system!

Using Interactive


4

1. Acquisition and coding of set of Requirements and Domain Knowledge 2. Apply A Prioritization Technique 3. Output of the ranking

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Problem Description (cont)

Find it! Why Ordered List? Order of implementation satisfy the developers’ constraints & delivers maximum value to the user.

Domain Knowledge: Prio & Dep

Using Interactive


5

Domain knowledge includes two precedence graphs • Prio • Dep

How to build them?

Prio: For 1st release, initial priorities

Dep: For 1st release, dependencies

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Prio

Dep

Classification: State-of-the-Art approaches

Using Interactive


6

Prioritization Approaches Classification

using user knowledge either performing pair-wise comparison or not

using domain knowledge

using both

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

User Knowledge refers to the awareness of the requirements attributes and the overall system functionalities to be developed

State-of-the-Art approaches (cont)

Using Interactive


7

Pairwise Comparison based approaches Analytic Hierarchy Process (AHP): involves comparing all unique pairs of requirements to determine which of the two is of higher priority, and to what extent Bubble Sort: compares two requirements & swap them if they are in the wrong order Cost-Value Approach: each individual requirement is determined on (i) the value to the users (ii) the cost of implementing the requirements. It uses the AHP technique Case Based Ranking (CBRank): exploits a machine learning algorithm to guide the elicitation of user preferences during the prioritization process

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions


Using Interactive


8

Non-Pairwise comparison based approaches Numerical Assignment: grouping requirements into different priority groups

MoScoW: groups all requirements into four priority groups MUST have, SHOULD have,

COULD have, and WON’T have.

Simple Ranking: requirements are simply ranked from integer 1 to N

Binary Search Tree: each node represents a requirement, requirements placed in the

left subtree of a node are of lower priority and those placed in the right subtree of a

node are of higher priority than the node priority.

$100 Method: each stakeholder is asked to assume having $100 to distribute over the

requirements in a ratio scale

Combining Techniques based approaches Planning Game: combination of two prioritization techniques i.e. Numerical Assignment

& Simple Ranking

Domain Knowledge based approaches Priority Groups: dividing requirements into separate groups. then groups are ranked by

using AHP

Genetic Algorithm: optimization is an application of GA & used in the problem of

requirements prioritization too; uses domain knowledge

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions


Using Interactive


9

Summary Approach Cons

AHP Scalability

Bubble Sort Scalability, time

CBRank unable to accept constraints i.e. Dep

Cost-Value Approach Time consuming

BST Sensitivity; a single error may build wrong tree

GA Can’t resolve contradictory;

$100 Method Longer; less confidence, biased

MoScoW Ambiguous final ordering

Simple Ranking unable handling complex scenarios

Scalability is a very common problem of all the approaches!

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Genetic Algorithm

Using Interactive


10

1. Acquisition and coding of set of Requirements and Domain Knowledge 2. Apply Genetic Algorithm (no user knowledge) 3. Output of the ranking (the most promising individual)

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Genetic Algorithm Pseudo Code

Using Interactive


11

The Canonical GA (a simple pseudo code is presented here): 1. choose initial population 2. evaluate each individual’s fitness REPEAT: 3. select best-ranking individuals to reproduce 4. apply crossover operator 5. apply mutation operator 6. evaluate each individual’s fitness until terminating condition (e.g. until at least one individual has the desired fitness or enough generations have passed)

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

What is Population & Individuals?

Using Interactive


12

Indv. ID Individual Disagree

Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >

Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >

Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >

Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >

Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >

Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >

Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >

Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >

Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >

Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >

R1 R3 R2 R4 R5 R6 R7 R8 R9

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Population!

A set of solution candidates

Pr1

An individual, also a solution candidate

A requirement as a gene

Using Interactive


13

How we define Individual’s Fitness?

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Domain Knowledge Encoding

Using Interactive


14

Transform the domain knowledge into graphs

Req Priorities Dependencies

R1 10 R2, R3, R7

R2 20 R3

R3 10

R4 20

R5 30 R4, R8

R6 40

R7 30 R3

R8 20

R9 50 R8, R5

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Disagreement Calculation

Using Interactive


15

Indv. ID Individual Disagree

Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >

Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >

Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >

Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >

Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >

Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >

Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >

Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >

Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >

Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >

R1 R3 R2 R4 R5 R6 R7 R8 R9

Conflicts = {(R5, R8), (R6, R7), (R6, R8), (R7, R8)}

Prio

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

For Pr1

Disagreement Calculation (cont)

Using Interactive


16

Indv. ID Individuals Disagree

Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >

Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >

Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >

Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >

Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >

Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >

Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >

Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >

Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >

Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >

R1 R3 R2 R4 R5 R6 R7 R8 R9

Conflicts = {(R1, R3), (R1, R7), (R1, R2), (R5, R8)}

Dep Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

For Pr1

Disagreement Calculation (cont)

Using Interactive


17


Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8

Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8

Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16

Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15

Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23

Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29

Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30

Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29

Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17

Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13

Total Conflicts = {(R5, R8), Prio (R6, R7), (R6, R8), (R7, R8), (R1, R3), Dep (R1, R7), (R1, R2), (R5, R8)} … and so on …

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

dis(pr1,pr2) {(r,s) pr1* | (r,s) pr2*}

Disagreement: Between a pair of ordering i.e. Pr1 and Prio, disagreement is the count of pairs that are inverted in two orderings. Lower disagreement defines higher fitness.

Formally define:

GA Crossover Operator

Using Interactive


18

We used cut-head/fill-in-tail and cut-tail/fill-in-head…

R2 R3

Positions 5-6 as cut points to cut-head/fill-in-tail

R6

Pr2

Pr3

Pr2’

R4 R1 R5 R8 R6 R7 R9

R5 R8 R1 R3 R7 R2 R9 R4 R6

R7 R8 R9 R2 R3 R4 R1 R5

- variation allows searching out different available niches, find better fitness values and subsequently better solutions - never produce chromosomes containing duplicate genes.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

GA Mutation Operator

Using Interactive


19

Mutation is basically a swap operator, we used

requirement-pair-swap

R2 R3

Pr2 R4 R1 R5 R8 R6 R7 R9

Pr2’

R2 R3 R6 R1 R5 R8 R4 R7 R9

- mutation causes movement in the search space - may produce a stronger chromosome.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

GA Selection Operator

Using Interactive


20

We used the Tournament Selection for our approach

- allows the selection pressure to be easily adjusted - faster than other selection operators i.e. Roulette Wheel Selection - better convergence

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Our IGA Approach

Using Interactive


21

1. Acquisition and coding of set of Requirements and Domain Knowledge

2. Apply Interactive Genetic Algorithm (exploiting User Knowledge)

3. Output of the ranking (the most promising individual)

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Our Approach: Pairs to be Elicited

Using Interactive


22


Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8

Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8

Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13

Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15

Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16

Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17

Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23

Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29

Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29

Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30

Ranked individuals w. r. t. disagreement

Pr1 = < R1, R3, R2, R4, R5, R6, R7, R8, R9 >

vs

Pr2 = < R2, R3, R4, R1, R5, R8, R6, R7, R9 >

TIE PAIRS

Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8)

Candidate pairs to be asked to the decision maker

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Our Approach: User Feedback

Using Interactive


23

Why ties appear in population? (i) Contradictory information w.r.t. initial constraints (ii) Nothing is expressed explicitly in initial constraints (iii) Common Domain Knowledge but different positions in the individuals Simple example: Why (R7, R8)? Case I: Contradictory w.r.t. Prio & Dep.. Why (R2, R3)? Case III: Common knowledge but different positions

TIE PAIRS

Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8) Prio

Dep

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Our Approach: User Feedback

Using Interactive


24

TIE PAIRS

Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8)

Eli

User Preference Graph eliOrd

Experience &

knowledge ‘<‘ or ‘>’

So, user knowledge is playing important role…

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Our Approach: New Round with New Constraints

Using Interactive


25

The new evolved population after using GA operators on population:

Crossover Mutation Selection

is compared against the new set of constraints graphs

Eli Dep Prio

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Using Interactive


26

Evaluation of the Approach

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

The Case Study

Using Interactive


27

Prioritize requirements for a real software system, as part of the project ACube (Ambient Aware Assistance)

designing a highly technological monitoring environment to be deployed in nursing homes to support medical and assistance staff

After user requirements analysis phase,

60 user requirements and 49 technical requirements

Four macro-scenarios have been identified.

ID Macro-Scenario # of requirements

FALL Monitoring falls 26

ESC Monitoring escapes 23

MON Monitoring dangerous behavior 21

ALL The three scenarios 49

*ACube is a social welfare project coordinated by Fondazione Bruno Kessler (FBK) and funded by Autonomous Province of Trento under Bando Grandi Progetti, 2006.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Gold Standard (GS)

Using Interactive


28

For each of the four macro-scenarios, we obtained the Gold Standard (GS) prioritization from the Software Architect of the ACube project

The GS prioritization is the ordering given by the software architect to the requirements when planning their implementation during the ACube project.

Why Gold Standard?

To measure disagreements with respect to GS.

To evaluate our approach in terms of disagreement against other non-interactive approaches using the same GS.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Using Interactive


29

Research Questions Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

R Q 1

Using Interactive


30

RQ1 (Convergence) Can we observe convergence with respect to the finally elicited fitness function?

- Convergence is not obvious immediately, as Eli Graph is evolving at early stages.

- Although the full fitness function is known only at end of elicitation process.

The best individual in each population converges toward a low value of the final fitness function. Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

YES!

R Q 2

Using Interactive


31

RQ2 (Role of interaction) Does IGA produce improved prioritizations compared to non-interactive requirement ordering?

IGA outperforms substantially GA (and RAND), especially when a higher number of pairwise comparisons can be carried out

YES!

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

R Q 3

Using Interactive


32

RQ3 (Role of initial precedence constraints) How does initial availability of precedence constraints affect the performance of IGA?

- Different type of Domain Knowledge affects IGA significantly - The improvement of IGA over GA is even higher when limited ranking information is available

Improves! Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

R Q 4

Using Interactive


33

RQ4 (Robustness) Is IGA robust with respect to errors committed by the user during the elicitation of pairwise comparisons?

IGA can tolerate user errors up to 20%.

YES! Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

General Discussion

Using Interactive


34

Cost/benefit trade off offered by IGA as compared to AHP is extremely interesting

With an elicitation effort reduced to 10% of the one required by AHP, IGA produces an apprx. ordering which has a quite low disagreement from the requirement positions in the GS.

User Errors tolerance offered by IGA as compared to AHP is highly reasonable

With an elicitation of very significant amount of less pairs than AHP (i.e. pairwise comparisons), it is reasonable even if IGA accepts up to 20% user error, while using AHP even 10% user error can result a total bad ordering.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Conclusions & Future Works

Using Interactive


35

We proposed an Interactive Genetic Algorithm to collect pairwise information useful to prioritize the requirements for a software system.

We also verified the robustness of the algorithm with respect to increasing user feedback errors.

We evaluated the approach in a real project (ACube).

In summary, we contributed A NOVEL APPROACH to prioritize requirements & tested its effectiveness empirically.

What’s Next? Algorithm:

- refining the algorithm - improving GA operators

Experiment - off-line: comparisons with other approaches - on-line: controlled experiments with real object (i.e. human)

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Paper Publication

Using Interactive


36

A paper was published from this work and was presented at SSBSE 2010: Paolo Tonella1, Angelo Susi1, and Francis Palma2, "Using Interactive GA for Requirements Prioritization" in 2nd International Symposium on Search Based Software Engineering 2010. 1Fondazione Bruno Kessler, Software Engineering Research Unit; 2Department of Inf. Eng. and Computer Science, University of Trento.

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Using Interactive


37

Thank You!

Outline

Problem

Related Works

Genetic Algo

Our Approach

Case Study

Results

Discussions

Conclusions

Date post:	11-Apr-2017
Category:	Technology
Upload:	francis-palma
View:	228 times
Download:	5 times

Using Interactive Genetic Algorithm for Requirements Prioritization

Technology