1
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
Evaluation of a Business Application Framework Using Complexity and
Functionality Metrics
Hikaru Fujiwara†, Shinji Kusumoto†, Katsuro Inoue†, Toshifusa Ootsubo‡, Katsuhiko Yuura‡
†Graduate School of Engineering Science,
Osaka University, Japan. ‡Business Solution Systems Division, Hitachi Ltd.
This research is partially supported
by International Information Science Foundation
2
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
3
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background (1/4)
• It becomes important to develop high-quality software cost-effectively.
• Reuse is one of the most famous techniques to attain it.• In object-oriented software development, developers
reuse a particular library called framework.• A framework is a collection of classes that provide a set
of services for a particular domain.
4
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background (2/4)
• A department of Hitachi Ltd. develops application software for many local governments.
• The developers have been using the original reuse technique based on a conventional module-based reuse.
5
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background (3/4)
• 47 prefectures in Japan.
• 3229 local governments, 671 cities, 1991 towns, 567 villages.
• A department of Hitachi Ltd. develops application software for many local governments.
• The requirement of the application is different depending on local ordinance.
6
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background (4/4)
• The department is going to introduce the framework in order to improve the efficiency.
• It is difficult to transfer the new framework to the development.
• To motivate the developers, we need to show the benefit of using framework quantitatively.
7
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
8
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
2. Approach2.1 Purpose
• We evaluate the usefulness of the framework quantitatively from two viewpoints.– saving cost– improving software quality
• We conducted two case studies to evaluate it.– Application : Applications for local governments– Language : Java
9
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
2.2 ReuseConventional module-based reuse (1/2)
• An application consists of a main module and some screen control modules.
• The main module controls business logics and screen transitions of the application.
• Each screen control module corresponds to one screen.
A : Database renewal program
X1 : Data InquiryY1 : Showing the
Inquiry ResultZ1 : Updating the
Record
(a) Health insurance application for a local government A
Screen transition controller
10
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
2.2 ReuseConventional module-based reuse (2/2)
A : Database renewal program
X1 : Data InquiryY1 : Showing the
Inquiry ResultZ1 : Updating the
Record
(a) Health insurance application for a local government A
Screen transition controller
X2 : Data InquiryY2 : Showing the
Inquiry ResultZ2 : Updating the
Record
(b) Health insurance application for a local government B
B : Database renewal program
Screen transition controller
11
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
2.2 ReuseFramework-based reuse
• The framework is intended to reuse the processing of the transition of the screens in addition to module-based reuse. – Typical transitions of screens are prepared.
• data inquiry, data renewal, data addition, data removal, etc.
Database Renewal Program A
Database renewal Framework
Data InquiryShowing the
Inquiry ResultUpdating the
Record
F1 : Framework
Specific Parameters to a Local
Government A
Specific Parameters to a Local
Government B
Database Renewal Program B
12
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
13
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
3. Evaluation Method
• We compare the framework-based reuse with the conventional module-based reuse from the viewpoints of cost and quality.(Case Study 1) Developing applications that have the same
functions.
Specification of function fa
framework-based
reuse
conventional
reuse
Ca
FW
Pa
FW : framework
14
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
3. Evaluation Method
(Case Study 2) Adding functions to the existing application.
Specification of function fa
Ca
FW
Pa
framework-based
reuse
conventional
reuse
Specification of function fb
Ca+b
FW
Pa+b
Specification of function fc
Ca+b+c
FW
Pa+b+c
15
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
3. Evaluation Method Metrics used in the case studies
• We could not collect the actual effort and the number of faults.
• We used following metrics to indirectly evaluate the productivity and quality.– OOFP (Object-Oriented Function Point) measures
functionality.
productivity.– C&K metrics (Chidamber and Kemerer’s metrics) measure
complexity.
quality.
16
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
3. Evaluation Method OOFP ( Object Oriented Function Points§)
• OOFP is an adaptation of FP(Function Point) to enable the measurement of object-oriented analysis and design specifications.– FP is measured from logical files(Internal Logical
Files:ILFs, External Interface Files:EIFs) and transactions (inputs, outputs, inquiries).
– OOFP is measured from logical files (ILF, EIF) and transactions (Service Requests:SRs).
• Classes correspond to logical files• Methods correspond to transactions
§:G.Caldiera, G.Antoniol, R.Fiutem, C.Lokan, “Definition and Experimental Evaluation of Function Points for Object-Oriented Systems”, IEEE, Proc. of METRICS98, pp.167-178 (1998).
17
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
3. Evaluation Method C&K metrics (Chidamber and Kemerer’s metrics)
viewpoints metrics how to calculate metrics of a class C
inheritance DIT the depth of C in the inheritance tree
NOC the number of immediate sub-classes subordinated to C
coupling CBO the number of couplings between C and any other class
RFC (the number of the methods in C) + (the number of the methods called by C)
method WMC the sum of the complexity of the methods in C
LCOM Assume that methods M1...Mn ∈ C,
Ii is a set of instance variables used in Mi,
(the number of pairs (Mk, Ml) such that Ik∩Il = φ)
- (the number of pairs (Mk, Ml) such that Ik∩Il≠φ)
18
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
19
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4. Case Study4.1 Case Study 1
• Four applications Ca, Cb, Cc, Cd are developed using the framework-based reuse.
• Four applications Pa, Pb, Pc, Pd are developed using the conventional reuse.
• Ci and Pi (i = a,b,c,d) implement the same function fi.
framework-based conventional
function
fa Ca Pa
fb Cb Pb
fc Cc Pc
fd Cd Pd
20
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1
• We compared Ci with Pi from the viewpoints of productivity and quality.
• We measured OOFP and C&K metrics from newly developed part of each application.
FW : framework : newly developed
Specification of function fa
Ca
FW
Pa
framework-based
reuse
conventional
reuse
FW : framework
21
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1Result of case study 1
number of classes
OOFP CBO RFC WMC LCOM
Ca
Cb
Cc
Cd
5
5
11
8
176
180
418
252
3.8
5.8
5.4
4.1
14.4
18.2
33.1
17.8
7.4
7.6
8.1
6.4
21.4
23.2
28.7
13.8
Pa
Pb
Pc
Pd
25
25
29
28
526
526
671
672
2.1
2.3
3.0
2.4
5.8
5.9
7.4
8.6
3.3
3.3
3.4
4.3
2.1
2.1
2.0
15.7
C&K metrics are the average values per class
22
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1Analysis of OOFP
framework-based (C)
conventional (P)
function
fa 176 526
fb 180 526
fc 418 671
fd 252 672
OOFP(Ci) < OOFP(Pi)
Application development using framework reduces the effort of development.
23
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1Analysis of OOFP
• To develop the framework, initial investment (effort) is needed.
• The OOFP of the framework FW is 1298.• The framework-based reuse is about 2.5 times more
effective than the conventional reuse.• The department develops the similar applications
repeatedly.• It will save the effort after three or four applications have
been developed, whereas the investment for the framework was spent.
24
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1Analysis of CBO and RFC
average CBO average RFC
framework-
based (C)
conventional
(P)
framework-
based (C)
conventional
(P)
function
fa 3.8 2.1 14.4 5.8
fb 5.8 2.3 18.2 5.9
fc 5.4 3.0 33.1 7.4
fd 4.1 2.4 17.8 8.6CBO(Ci) > CBO(Pi), RFC(Ci) > RFC(Pi)
All of the methods called by Ci are included in the framework. If the framework is high quality, the complexity does not affect the quality of the overall application program.
25
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.1 Case Study 1Analysis of WMC and LCOM
average WMC average LCOM
framework-
based (C)
conventional
(P)
framework-
based (C)
conventional
(P)
function
fa 7.4 3.3 21.4 2.1
fb 7.6 3.3 23.2 2.1
fc 8.1 3.4 28.7 2.0
fd 6.4 4.3 13.8 15.7WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi)
There are many simple methods, that set/get the values of the attribute.
The complexity does not affect the quality of the overall application program.
26
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
27
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2• At first, application Ca and Pa are developed.
• Function fb, fc, fd are continuously added to Ca and Pa.
• Ci and Pi (i = a,a+b,a+b+c,a+b+c+d) implement the same function fi.
framework-based
conventional
function
fa Ca Pa
fa+b Ca+b Pa+b
fa+b+c Ca+b+c Pa+b+c
fa+b+c+d Ca+b+c+d Pa+b+c+d
28
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2• We compared the differences between the two successive
applications from the viewpoint of productivity and quality.• We measured OOFP and C&K metrics from newly developed
part of each application.
Specification of function fa
Ca
FW
Pa
framework-based
reuse
conventional
reuse
Specification of function fb
Ca+b
FW
Pa+b
Specification of function fc
Ca+b+c
FW
Pa+b+c
29
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2Result of case study 2
number of classes
OOFP CBO RFC WMC LCOM
Ca
Ca+b
Ca+b+c
Ca+b+c+d
5
7
15
20
176
251
578
743
3.8
5.0
5.9
5.8
14.4
16.7
30.1
28.3
7.4
7.6
8.3
7.9
21.4
20.1
28.6
25.6
Pa
Pa+b
Pa+b+c
Pa+b+c+d
25
29
39
46
526
615
849
1084
2.1
2.8
3.7
4.0
5.8
6.9
8.6
10.5
3.3
3.3
3.3
3.9
2.1
2.0
1.8
10.0
C&K metrics are the average values per class
30
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2Analysis of OOFP
framework-based (C)
conventional (P)
function
fa 176 526
fa+b 251 615
fa+b+c 578 849
fa+b+c+d 743 1084
75
327
165
89
234
235
OOFP(Ca+b+c - Ca+b) > OOFP(Pa+b+c - Pa+b)
The adaptability of the framework to the function fc is not good. It is necessary to add new components for the functions like fc to the framework.
31
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2Analysis of OOFP
OOFP(Ca+b+c+d) < OOFP(Pa+b+c+d)
framework-based (C)
conventional (P)
function
fa 176 526
fa+b 251 615
fa+b+c 578 849
fa+b+c+d 743 1084
The framework-based reuse is more effective to reduce the effort of development than the conventional reuse.
32
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
4.2 Case Study 2Analysis of CBO and RFC
average of CBO average of RFC
framework-based (C)
conventional (P)
framework-based (C)
conventional (P)
function
fa 3.8 2.1 14.4 5.8
fa+b 5.0 2.8 16.7 6.9
fa+b+c 5.9 3.7 30.1 8.6
fa+b+c+d 5.8 4.0 28.3 10.5
1.2
0.9
-0.1
0.9
0.9
0.3
2.3
13.4
-1.8
1.1
1.7
1.9
RFC(Ca+b+c - Ca+b) > RFC(Pa+b+c - Pa+b)
Ca+b+c calls a lot of methods included in the framework, in order to handle many data items in the function fc. If the framework is high quality, the complexity does not affect the overall application program.
33
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
1. Background
2. Approach 2.1 Purpose
2.2 Reuse
3. Evaluation Method
4. Case Study 4.1 Case Study 1
4.2 Case Study 2
5. Conclusion
Contents
1. Background
34
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
5. Conclusions• We have experimentally evaluated the usefulness of the
framework quantitatively from the viewpoints of quality and saving cost.
• As the result of the case studies, the framework-based reuse is more effective than the conventional reuse.
• In order to show the usefulness of the framework, we are going to apply the framework to many software development projects in future.
35
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
36
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
Appendix How to measure OOFP(1)
OOFP=OOFPILF+OOFPEIF+OOFPSR
• For each ILF, OOFPILF is measured using DETs (Data Element Types) and RETs (Record Element Types).
• For each EIF, OOFPEIF is measured using DETs and RETs.
• For each SR, OOFPSR is measured using DETs and FTRs (File Types Referenced).
• Finally, OOFP is calculated by summing up OOFPILF, OOFPEIF and OOFPSR.
37
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
AppendixHow to measure OOFP (2)
• To calculate OOFP from Java source code, we suppose that the concepts in Java correspond to ones in OOFP as follows.
OOFP Java
Logical files ILFs Classes within the application.
EIFs Classes outside of the application.
DETs Number of simple attributes (such as integers, strings)
RETs Number of complex attributes (such as objects)
Transactions SRs Methods within the application.
DETs Number of simple arguments, instance variables and class variables.
FTRs Number of complex arguments, instance variables, class variables and objects.
38
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
AppendixCase Study 2; Analysis of WMC and LCOM (1)
average of WMC average of LCOM
framework-based (C)
conventional (P)
framework-based (C)
conventional (P)
function
fa 7.4 3.3 21.4 2.1
fa+b 7.6 3.3 20.1 2.0
fa+b+c 8.3 3.3 28.6 1.8
fa+b+c+d 7.9 3.9 25.6 10.0
0.2
0.7
-0.4
0.0
0.0
0.6
-1.3
8.5
-3.0
-0.1
-0.2
8.2
LCOM(Ca+b+c - Ca+b) > LCOM(Pa+b+c - Pa+b)There are a lot of set/get methods in Ca+b+c , in order to handle many data items in the function fc.
39
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
AppendixCase Study 2; Analysis of WMC and LCOM (2)
average of WMC average of LCOM
framework-based (C)
conventional (P)
framework-based (C)
conventional (P)
function
fa 7.4 3.3 21.4 2.1
fa+b 7.6 3.3 20.1 2.0
fa+b+c 8.3 3.3 28.6 1.8
fa+b+c+d 7.9 3.9 25.6 10.0
0.2
0.7
-0.4
0.0
0.0
0.6
-1.3
8.5
-3.0
-0.1
-0.2
8.2
LCOM(Ca+b+c+d - Ca+b+c) < LCOM(Pa+b+c+d - Pa+b+c)There are complex transitions of the screens in function fd. In Pa+b+c+d , the LCOM values of the classes implement these transactions are high.
40
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
AppendixCase Study 2; Analysis of WMC and LCOM (3)
average of WMC average of LCOM
framework-based (C)
conventional (P)
framework-based (C)
conventional (P)
function
fa 7.4 3.3 21.4 2.1
fa+b 7.6 3.3 20.1 2.0
fa+b+c 8.3 3.3 28.6 1.8
fa+b+c+d 7.9 3.9 25.6 10.0
WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi)
We examined the classes of Ci that have high WMC and LCOM values. It is found that there are many simple methods, that set/get the values of the attribute.
41
Software Engineering Research Group, Graduate School of Engineering Science, Osaka University
AppendixProcess of Case Study
• Development of the framework• Case Study1
– Two developers
– Develop Ca, Cb, Cc, Cd
– Develop Pa, Pb, Pc, Pd
• Case Study2– Two developers
– Develop Ca, Ca+b, Ca+b+c, Ca+b+c+d
– Develop Pa, Pa+b, Pa+b+c, Pa+b+c+d
• Apply the metrics to the applications.• Analysis