Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor...

Test process analysis ofGateway GPRS Support NodeAuthor: Jorma Tuominen

Supervisor: Professor Sven-Gustav Häggman

Nokia Networks

17.1.2006 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 2 (15)

Agenda

• Introduction• GGSN in GPRS and 3G packet core network• GGSN in Nokia Intelligent Content Delivery

system • Software development process and testing

phases

• Research problem

• Research method

• Results

• Conclusions


GPRS and 3G packet core network

BSC

RNC3G SGSN

2G SGSN

NMS

BG

GGSN

LIG

Internet

FW

FW

Corporatecustomer

Mobile ISP

Intranetservices

WAPMMS

IP backbone

CG

Inter-PLMN

network

BTS

BS

NewsSportsGames

Trading


Nokia Intelligent Content Delivery System ICD

GGSN TA

CG OSC NSM

user data user data

user dataCA

GTP’ RADIUS orDIAMETER

LDAP

LDAP

user data

+ RADIUS


SW development process and testing phases

Product Architecture Design

Functional Specifications

Implementation

Detailed Design & Specifications

Product Requirements

Customer/Business Requirements

Network/System Requirements

Customer Acceptance

System VerificationSyVe

Functional Testing FT

Product IntegrationPI

Module TestingMT

System Testing ST

System Program

Product Program

System Integration



Implementation





Customer Acceptance




Module TestingMT

System Testing ST

System Program

Product Program

System Integration


Research problem

Testing requires tradeoffs between cost, time

and quality.

How available testing time and resources can be used as effectively as possible to find and correct the critical defects from the product before it is released?


Research method

• Participating in GGSN system testing

• Literature study of GPRS and 3G packet core network elements and protocols

• Literature study of software process and software testing

• Collecting and analyzing testing metrics

• Defect analysis

• Risk analysis

• Planning risk based testing


Testing metrics

• Testing effort metrics• Distribution of working hours by test phases

• Defect metrics• Number of discovered defects per week• Percentage of “correction not needed” defect

reports• Test case metrics

• Number of test cases• Percentage of not relevant test cases

• Testing effectiveness metrics• Hours needed to discover one defect• Number of test cases needed to discover one

defect• Defect Removal Effectiveness


Testing metricsexamples

• Distribution of working hours by test phases(lessons learned)

• Number of discovered defects per week(real-time metrics)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

PI FT ST

Rel1

Rel2

Rel3

0

2

4

6

8

10

12

14

16

18

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

PI

FT

ST

Cu

E3 E3.5

E2

E4


Defect analysis

• Search for the error prone functionality areas• Defects discovered in the earlier product releases• Defects discovered in the earlier testing phases• Defects discovered by the customers

• Can be done after product release or during testing• Pareto 80-20 phenomena

• Many software phenomena follow a Pareto distribution: 80% of contribution comes from 20% of the contributors [Boehm 1989]

• Examples:• 20% of the software modules contribute 80% of

the errors.• 20% of the errors cause 80% of the down time


Risk analysis• System requirements must be prioritised• What is the likelihood that this feature will fail to

operate?• What would be the impact on the user if this feature

will fail to operate? • The likelihood of failure depends on the following:

• error proneness of the requirement• newness of the requirement• complexity of the requirement

• Likelihood of failure: 1 = very high, ... 5 = very low• Impact on the user: 1 = severe, ... 5 = negligible

• Risk priority = (Likelihood of failure) x (Impact on the user)


Risk based testing

Risk

100%

50%

Testing effortE2

E1

Risk based testing

Traditional testing


Results

• Requirements have been divided into three groups:• High, medium and low risk requirements

• Test case titles will be planned for each requirement• Test cases will be designed and executed as follows:

• High risk requirements: 100% of test cases• Medium risk requirements: 70% of test cases • Low risk requirements: 30 % of test cases

• Test design and execution effort can be reduced about 30%.

• Effect on the quality of the product can not be estimated yet, because the product is not yet released.


Conclusions and future work

• The availability of the real-time testing metrics should be improved.

• Co-operation between the product development, the testing and the customer support should be strengthened.

• The most important future work is to harmonize the usage of requirement management database, test case database and defect tracking database.


Thank You!

Questions?


Additional slides


Cost of defectsPhase where defect is

discovered

Relative cost to correct a defect

Requirements 1

Design 3-6

Coding 10

Development testing 15-40

Acceptance testing 30-70

Production 40-1000



Implementation





Customer Acceptance




Module TestingMT

System Testing ST

System Program

Product Program

System Integration



Implementation





Customer Acceptance




Module TestingMT

System Testing ST

System Program

Product Program

System Integration

Phase where defect is discovered

Relative cost to correct a defect

Definition 1

High-level design 2

Low-level design 5

Code 10

Unit test 15

Integration test 22

System test 50

Post delivery 100+


Early involvement of the testers• Important aspects of early involvement of the

testers are: “Testers need a solid understanding of the product so they can devise better and more complete test plans, designs, procedures, and cases. Early test-team involvement can eliminate confusion about functional behavior later in the project lifecycle. In addition, early involvement allows the test team to learn over time which aspects of the application are the most critical to the end user and which are the highest-risk elements. This knowledge enables testers to focus on the most important parts of the application first, avoiding over-testing rarely used areas and under-testing the more important ones.” [Dustin 2003].


Thoughts about testing

• “Program testing can be used to show the presence of bugs, but never to show their absence!” [Dijkstra 1969].

• Testing completion criteria:Risk of stopping testing = Risk of continuing testing

• “In most cases ‘what’ you test in a system is much more important than ‘how much’ you test” [Craig 2002].

• “Prioritise tests so that, when ever you stop testing, you have done the best testing in the time available” [ISEB 2003].

• No risk No test


References• Myers G. The art of software testing, USA, John Wiley & Sons, ISBN 0-471-04328-1, 1979,

177 p.

• Hetzel B. Complete Guide to Software Testing, USA, John Wiley & Sons, ISBN 0-471-56567-9, 1988, 284 p.

• Craig R., Jaskiel S. Systematic Software Testing, USA, Artech House Publishers, ISBN 1-58053-508-9, 2002, 536 p.

• Dijkstra E. W. Structured programming. Proceedings of the Second NATO Conference on Software Engineering Techniques, Rome Italy, NATO, 1969, pp. 84-88.

• Dustin E. Effective Software Testing, USA, Addison-Wesley, ISBN 0-201-79429-2, 2003, 271 p.

• Bender R. Requirements Based Testing presentation. Proceedings of StarEast 2005, Software testing analysis & review conference, USA Orlando, Bender RBT Inc. 2005, 232 p.

• Beizer B. Black-Box Testing: Techniques for Functional Testing of Software and Systems, USA, John Wiley & Sons, ISBN 0-471-12094-4, 1995, 294 p.

• Koomen T., Pol M. Test Process Improvement: A practical step-by-step guide to structured testing, Great Britain, Addison-Wesley, ISBN 0-201-59624-5, 1999, 218 p.

• Hutcheson M. Software Testing Fundamentals: Methods and Metrics, USA, John Wiley & Sons, ISBN 0-471-43020-X, 2003, 408 p.

• Boehm B. W. Software Risk Management, USA, IEEE Press, ISBN 0-8186-8906-4, 1989, 495 p.

• Grove Consultants, ISEB testing foundation course material, Great Britain, Grove Consultants, 2003, 98 p.

Date post:	01-Apr-2015
Category:	Documents
Upload:	johanna-kennington
View:	219 times
Download:	2 times

Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor...

Documents