Requirements Engineering Maturity Requirements Engineering Maturity Measurement and EvaluationMeasurement and EvaluationMeasurement and Evaluation Measurement and Evaluation

(A Case Study of Bank X in Indonesia)

Bahana Wiradanti Bahana Wiradanti & Rajesri Govindaraju& Rajesri Govindaraju

Industrial Engineering DepartmentInstitut Teknologi Bandung

International Conference on Industrial and Intelligent Information 2011 (ICIII)

International Conference on Industrial and Intelligent Information 2011 (ICIII)Intelligent Information 2011 (ICIII)1 – 3 April 2011, Bali, IndonesiaIntelligent Information 2011 (ICIII)1 – 3 April 2011, Bali, Indonesia

2www.slideshare.net/wiradantib

ContentsContentsContentsContents

1 Introduction1. Introduction2. Methodology3. Conclusion 4. Future Research

3www.slideshare.net/wiradantib

1. Introduction1. Introduction

Research BackgroundResearch BackgroundResearch BackgroundResearch Background

Increase in needs for information system

5IS development needs to be managedwww.slideshare.net/wiradantib

Research BackgroundResearch BackgroundResearch BackgroundResearch Background

REQUIREMENTSREQUIREMENTSREQUIREMENTSREQUIREMENTSIncrease in needs for information system????

Errors in system requirementscould be up to 100x

More expensive to fixThan errors in system implementationy pBoehm (1983) in Sommerville& Ransom(2005)

6IS development needs to be managedwww.slideshare.net/wiradantib

system requirements define what must be y qdone by information systems, or any property or quality of what should be provided by the y ysystem (Bentley & Whitten, 2007, p. 208)

Requirements Engineering Requirements Engineering (RE)(RE)q g gq g gProcessProcess

“The goal is to create andThe goal is to create and maintain a system requirements document.”

(Sommerville, 2006, p.143)

7Better quality, in budget, in time

www.slideshare.net/wiradantib

Bank XBank XBank XBank X• Bank X is an important organization in Indonesia’s economy. • They have been executing many information system• They have been executing many information system

development projects.

IMWork Units

IM Unit IT Unit

(fi l )(Initial Req) (Req Management)

(final req)

Problems :Problems :• IM Unit felt their RE process has already been in the right

direction, has implemented ISO 9001.

8• Systems that have been made are not satisfying stakeholders.• Increase in completion time and cost.www.slideshare.net/wir

adantib

Research QuestionResearch QuestionResearch QuestionResearch Question“How mature is Bank X in theirHow mature is Bank X in their

Requirements Engineering process?

WWhich RE practices has been well defined and consistentlydefined and consistently implemented, also which are still weak in Bank X? ”

P f t dPurpose of case study:Not only helping improvements in Bank X’s RE process, b t also to learn from their e perience

9

but also to learn from their experiencewww.slideshare.net/wir

adantib

2 Methodology2 Methodology2. Methodology2. Methodology

Research ModelResearch ModelResearch ModelResearch Model

Requirements Engineering Maturity Measurement Framework (REMMF)

With3 Dimensions of Motorola3 Dimensions of Motorola

(Niazi et.al, 2008)

Basic, intermediate and d d tiadvanced practices

Approach Deployment Resultspp p y

11www.slideshare.net/wiradantib

Research ModelResearch Model (Cont )(Cont )

REMMF (Ni i t l 2008)

Research Model Research Model (Cont.)(Cont.)

REMMF (Niazi et.al, 2008)

Strength & WeaknessOrgaization

Maturity Level gof organization based on each

Maturity Level

Good Practice score

(10 i t l )Level2Level3 –Defined

(10 point scale)

Level1 –

Level2 –Repeatable

Level1 –Initial

12www.slideshare.net/wiradantib

Maturity Measurement ModelMaturity Measurement ModelMaturity Measurement ModelMaturity Measurement Model

Requirements Engineering Maturity Measurement F k (REMMF)Framework (REMMF)

With3 Dimensions of Motorola

3: standardized guideline2: normal use1 di ti (Niazi et.al, 2008)

10 Point scale1: discretionary use0: never used

13www.slideshare.net/wiradantib

Motorola’s Instrument (Daskalantonakis, 1994 in Niazi et al, 2008)

Score APPROACH DEPLOYMENT RESULTS

Poor (0)No commitment, management

t d i ti bilitNo part of the organization uses the

tiNo results, ineffective

Poor (0) support and organization ability practice

Weak (2)Management recognizes the needs to implement, but no

commitment&support

Fragmented and inconsistent use Spotty and inconsistent results

commitment&support

Fair (4)

There are management recognition, support and ability,

wide but not complete i b

Less fragmented use and some consistency, deployed in some major parts

of the organization

Consistent and positiveresults for several parts of

the organizationcommitment by management

Marginally Qualified (6)

Much wider approach, and some of the management have

been proactive

Deployed in many parts of the Organization, mostly consistent use, and monitoring/verification of use for many

Positive measurable results and consistent in

mostQ ( ) p g yparts of the organization parts of the organization

Q lifi d (8)

Total management commitment and majority of management is

ti

Deployed and consistently used in almost all parts of the organization, monitoring/

ifi ti f f l t ll t f

Positive measurable results and consistent in

l t llQualified (8) proactive verification of use for almost all parts of the organization, and there is a mechanism

to distribute the lessons learned

almost allparts of the organization

Management provides zealous Pervasive and consistent deployed, Requirements exceeded

Outstanding (10)

g pleadership and commitment,

organizational excellence in the practice recognized even outside the organization

p yconsistent use over time across all parts of

the organization, and monitoring/verification for all parts

of the organization

qconsistently world-class

results, counsel sought by others

outside the organization of the organization

14www.slideshare.net/wiradantib

More Measurement:More Measurement:Agency Y (comparison organization) to see whether the assessment model could distinguish Good Practices score of organizations, which maturity level are different

Data Collection & ProcessingData Collection & Processinggg• Two kinds of data:

Primary: questionnaires data– Primary: questionnaires data– Secondary: requirements document

• Data processing is done according to previous studiesData processing is done according to previous studies.

15www.slideshare.net/wiradantib

3. Conclusion3. Conclusion

ConclusionConclusionConclusionConclusion1. Bank X:

Maturity Level : Level 3RE Process– Strengh : “requirements document “ and

“requirements elicitation”W k– Weakness :

whole good practice score still weak Implementing intermediate and advanced practices Implementing intermediate and advanced practices

2. ComparisonBank X Agency YBank X Agency Y

Maturity Level

• Level 3 • Level 1

Strength&

• the whole good practice score < 7• Difficulties : Implementing

intermediate and advanced

• the whole good practice score < 3• Difficulties : Implementing basic

practices

17Weakness

intermediate and advancedpractices

practices

www.slideshare.net/wiradantib

Further Conclusion & Lessons LearnedFurther Conclusion & Lessons Learned• This case study brings improvement areas: priority in

Further Conclusion & Lessons LearnedFurther Conclusion & Lessons Learned

improving good practices

• The assessment model was considered easy to use

G l l t lt di i• General lowest score: results dimension score Lessons Learned: Communication problems between

RE analysts &system builder are common:RE analysts &system builder are common:Uphold group decision making, esp in elicitation

process. pMore participation or involvement of system

builders. Create more awareness to the RE process.

18www.slideshare.net/wiradantib

4. Future Research4. Future Research• Survey to measure the RE process performed in

Indonesian enterprises. • Improve assessment framework (communication

aspect)aspect)

Thank YouThank You

20www.slideshare.net/wiradantib