Chapter 1

1

Chapter 1

The Software Quality Challenge

2

The uniqueness of software quality assurance

DO you think that there is a bug-free software?

Can software developers warrant their software applications and their documentation from any bugs or defects ?

What are the essential elemental differences between software and other industrial products such as automobiles, washing machines etc?

3

The essential differences between software and other industrial products can be categorized as follows :

1. Product complexity : # of operational modes the product permit.

2. Product visibility : SW products are invisible.

3. Product development and production process.

4

The phases at which the possibility of detecting defects in industrial products and software products:

SW products do not benefit from the opportunities for detection of defects at the three phases of the production process

Industrial products: Product development : QA -> product prototype Product production planning : Production - line Manufacturing : QA procedure applied Software products: Product development : QA -> product prototype Product production planning : Not required Manufacturing : Copying the product & printing copies

5

Factors affecting detecting defects in SW products VS other industrial products:

Characteristic SW products Other industrial products

Complexity Usually, v. complex allowing for v. large number of operational options

Degree of complexity much lower

Visibility Invisible, impossible to detect defects or omissions by sight ( diskette or CD storing )

Visible, allowing effective detection of defects by sight

Nature of development and production process

Opportunities to detect defects arise in only one phase, namely product development

Opportunities to detect defects arise in all phases of development and production

6

Important Conclusion

The great complexity as well as invisibility of software, among other product characteristics, make the development of SQA methodologies and its successful implementation a highly professional challenge

7

Pupils & students Hobbies Engineers, economics , mgt & other fields SW development professionals

All those SW developers are required to deal with SW quality problems “Bugs”

The environment for which SQA methods are developed

8

SQA environmentThe main characteristics of this environment :

1. Contractual conditions

2. Subjection to customer-supplier relationship

3. Required teamwork

4. Cooperation and coordination with other SW teams

5. Interfaces with other SW systems

6. The need to continue carrying out a project despite team member changes.

7. The need to continue out SW maintenance for extended period.

9

Contractual conditions

the activities of SW development & maintenance need to cope with :

A defined list of functional requirements The project budget The project timetable

10

Subjection to customer-supplier relationship

SW developer must cooperate continuously with customer : To consider his request to changes To discuss his criticisms To get his approval for changes

11

Required teamwork

Factors motivating the establishment of a project team: Timetable requirements The need of variety The wish to benefit from professional

mutual support & review for enhancement of project quality

12

Cooperation and coordination with other SW teams

Cooperation may be required with: Other SW dev. Teams in the same org. HW dev. teams in the same org. SW & HW dev. teams of other suppliers Customer SW and HW dev. teams that take part

in the project’s dev.

13

Interfaces with other SW Systems

Input interfaces Output interfaces I/O interfaces to the machine’s control board,

as in medical and lab. Control systems

14

The need to continue carrying out a project despite team member changes.

During project dev. Period we might be face : Leave from the members of the team Switch in employees Transfer to another city

15

The need to continue out SW maintenance for extended period.

From 5 to 10 years , customers need continue to utilizing their systems: Maintenance Enhancement Changes ( Modification )

16

Chapter 2

What is Software Quality ?

17

What is Software ?

IEEE Definition:

Software Is :

Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.

18

IEEE Definition is almost identical to the ISO def. ( ISO/IEC 9000-3 )

Computer programs (“Code”) Procedures Documentation Data necessary for operation the

SW system.

19

TO sum up:

Software quality assurance always includes :

Code quality The quality of the documentation And the quality of the necessary SW

data

20

SW errors, faults and failures

Questions arise from HRM conference Page 16.

An error : can be a grammatical error in one or more of the code lines, or a logical error in carrying out one or more of the client’s requirements.

Not all SW errors become SW faults. SW failures that disrupt our use of the software.

21

The relationship between SW faults & SW failures:

Do all SW faults end with SW failures? The answer is not necessarily The SW fault becomes a SW failure only when it is

activated.

Example page 17-18

22

Classification of the causes of SW errors

SW errors are the cause of poor SW quality SW errors can be

Code error Documentation error SW data error

The cause of all these errors are human

23

The nine causes of software errors

1. Faulty requirement definition2. Client-developer communication failures3. Deliberate deviation from SW requirements4. Logical design errors5. Coding errors6. Non-compliance with documentation and coding

instructions7. Shortcomings of the testing process8. Procedure errors9. Documentation errors

24

Faulty requirement definition

1. Erroneous definition of requirements

2. Absence of vital requirements

3. Incomplete definition of requirements

4. Inclusion of unnecessary requirements

25

Client-developer communication failures

Misunderstandings resulting from defective client-developer comunications.

Misunderstanding of the client’s requirements changes presented to the developer

In written forms Orally Responses to the design problems others

26

Deliberate deviation from SW requirements

The developer reuse SW modules taken from the earlier project

Due to the time budget pressure Due to the unapproved improvements

27

Logical design errors

This is come from systems architects, system analysts, SW engineers such as:

Erroneous algorithms Process definitions that contain sequencing

errors Erroneous definition of boundary conditions Omission of required SW system states Omission of definitions concerning reactions to

illegal operations

28

Coding errors

Misunderstanding the design documentation Linguistic errors in the prog. Lang. Errors in the application of CASE and other

dev. Tools etc

29

Non-compliance with documentation and coding

Team members who need to coordinate their own codes with code modules developed by non-complying team members

Individuals replacing the non-complying team member will find it difficult to fully understand his work.

Design review to other non-complying team

30

Shortcomings of the testing process

Incomplete testing plans Failures to document and report errors and

faults Failures to promptly correct detected SW

faults as a result of inappropriate indications of the reasons for the fault.

Incomplete correction of detected errors.

31

Procedure errors & documentation errors

See example page 22

32

Software quality - Definition IEEE

1. The degree to which a system, component, or process meets specified requirements.

2. The degree to which a system, component, or process meets customer or user needs or expectations.

33

Software Quality Pressman’s def.

Conformance to explicitly stated functional and performance requirements, explicitly documented standards, and implicit characteristics that are expected of all professionally developed software.

34

Software Quality Assurance The IEEE Definition

SQA is :

1. A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.

2. A set of activities designed to evaluate the process by which the products are developed or manufactured. Contrast with quality control.

35

IEEE SQA definition – exclude the maintenance & timetable and budget issues.

The Author adopts the following :

SQA should not be limited to the development process. It should be extended to cover the long years of service subsequent to product delivery. Adding the software maintenance functions into the overall conception of SQA.

SQA actions should not be limited to technical aspects of the functional requirements, It should include activities that deal with scheduling and timetable and budget .

36

SQA – Expanded Definition

.

This definition corresponds strongly with the concepts at the foundation of ISO 9000-3, 1997and also corresponds to the main outlines of the CMM for softwareSee the Table 2.2 page 27

A systematic, planned set of actions necessary to provide adequate confidence that the software development process or the maintenance of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating within the budgetary confines.

37

Software Quality Assurance Vs. Software Quality Control

Quality Control : a set of activities designed to evaluate the quality of a developed or manufactured product. It take place before the product is shipped to the client.

Quality Assurance : the main objective is to minimize the cost of guaranteeing quality by a variety of activities performed throughout the causes of errors, and detect and correct them early in the dev. Process.

38

The objectives of SQA activitiessee page 29

Software development ( process-oriented )

Software maintenance ( Product-oriented )

39

SQA Vs Software Engineering

SW Engineering ( IEEE def. )

1. The application of a systematic, restricted, quantifiable approach to the development and maintenance of SW; that is the application of engineering to software.

40

Chapter 3

Software Quality Factors

41

SQ. Factors

From the previous chapters we have already established that the requirements document is one of the most important elements for achieving SQ.

What is a “Good” SQ requirements document ?

42

The need for comprehensive SQ requirements

Our Sales IS seems v. good , but it is frequently fails, at least twice a day for 20 minutes or more.( SW house claims no responsibility….

Local product contains a SW and every thing is ok, but, when we began planning the development of a European version, almost all the design and programming will be new.

etc see page 36.

43

There are some characteristics common to all these buts :

All SW projects satisfactorily fulfilled the basic requirements for correct calculations.

All SW projects suffered from poor performance in important areas such as maintenance, reliability, SW reuse, or training.

The cause for poor performance of the developed SW projects in these areas was lack of predefined requirements to cover these important aspects of the SW functionality.

The solution is :The need for a comprehensive definition of requirements

( SQ Factors )

44

Classification of SW requirements into SW quality factors.

McCall’s Factor Model This model classifies all SW requirements into 11 SW quality

factors, grouped into 3 categories: Product operation: Correctness, Reliability, Efficiency,

Integrity, Usability Product revision : Maintainability, Flexibility, Testability Product transition : Portability, Reusability,

Interoperability.

See the McCall model of SW quality factors tree

see page 38

45

Product operation SW quality factors

Correctness: Output specifications are usually multidimensional ; some common include: The output mission The required accuracy The completeness The up-to-dateness of the info. The availability of the info.( the reaction time ) The standards for coding and documenting the SW system See Example page 39.

46


Reliability:

Deals with failures to provide service. They determine the maximum allowed SW system failure rate, and can refer to the entire system or to one or more of its separate functions.

See examples page 39 ( heart-monitoring unit )

47


Efficiency:

Deals with the HW resources needed to perform all the functions of the SW system in conformance to all other requirements.

See examples page 40 ( CPU speed .. etc ) Integrity:

Deals with the SW system security, that is requirements to prevent access to unauthorized persons.

See examples page 40

48


Usability:

Deals with the scope of staff resources needed to train a new employee and to operate the SW system.

See examples page 41

49

Product revision SW quality factors

Maintainability :

Maintainability requirements determine the efforts that will be needed by users and maintenance personnel to identify the reasons for SW failures, to correct the failure, and to verify the success of the corrections.

Example : Typical maintainability requirements:

1. The size of a SW module will not exceed 30 statements

2. The programming will adhere to the company coding standards and guidelines.

50


Flexibility :

The capabilities and efforts required to support adaptive maintenance activities are covered by flexibility requirements. This factor’s requirements also support perfective maintenance activities, such as changes and additions to the SW in order to improve its service and adapt it to changes in the firm’s technical or commercial environment.

Example :page 42

51


Testability : - Deal with the testing of an IS as well as with its

operation.- Providing predefined intermediate results and log files.- Automatic diagnostics performed by the SW system

prior starting the system, to find out whether all components of SW system are in working order.

- Obtain a report about detected faults.

Example :page 42, 43

52

Product transition SW quality factors

Portability : - Tend to the adaptation of a SW system to other

environments consisting :- Different HW- Different OS

Example : SW designed to work under windows 2000 env. Is required to allow low-cost transfer to Linux.

53


Reusability : - Deals with the use of SW modules originally designed

for one project in a new SW project currently begin developed.

- The reuse of SW is expected to save resources., shorten the project period, and provide higher quality modules. These benefits of higher quality are based on the assumption that most SW faults have already been detected by SQA activities performed previously on it.

54


Interoperability : - Focus on creating interfaces with other SW systems or

with other equipment firmware.- Example:

- The firmware of medical lab. equipment is required to process its results according to a standard data structure that can be then serve as input for a number of standard laboratory IS.

55

Alternative Models Of SW Quality Factors

Two other models for SQ factors: Evans and Marciniak 1987 ( 12 factors ) Deutsch and Willis 1988. ( 15 factors )

Five new factors were suggested Verifiability Expandability Safety Manageability Survivability

56

Alternative Models Of SW Quality Factors

Five new factors were suggested Verifiability: define design and programming features that enable

efficient verification of the design and programming ( modularity, simplicity, adherence to documentation and prog guidelines. )

Expandability: refer to future efforts that will be needed to serve larger populations, improve services, or add new applications in order to improve usability.

Safety: meant to eliminate conditions hazardous to equipment as a result of errors in process control SW.

Manageability: refer to the admin. tools that support SW modification during the SW development and maintenance periods.

Survivability: refer to the continuity of service. These define the minimum time allowed between failures of the system, and the maximum time permitted for recovery of service.

57

Who is interested in the definition of quality requirements ?

The client is not the only party interested in defining the requirements that assure the quality of the SW product.

The developer is often interested also specially : Reusability Verifiability Porotability

Any SW project will be carried out according to 2 requirements document : The client’s requirements document The developer’s additional requirements document.

58

Chapter 4

The Components Of the SQA system- Overview

59

The SQA system- an SQA architecture

SQA system components can be classified into 6 classes : Pre-project components Components of project life cycle activities assessment Components of infrastructure error prevention and

improvement. Components of SQ management Components of standardization, certification, and SQA

system assessment Organizing for SQA- the human components

60

Pre-project Components :

To assure that :

1. The project commitments have been adequately defined considering the resources required, the schedule and budget.

2. The development and quality plans have been correctly determined.

61

Components of project life cycle activities assessment:

The project life cycle composed of two stages:1. The development Life cycle stage:

Detect design and programming errors Its components divided into:

Reviews Expert opinions Software testing Assurance of the quality of the subcontractors’ work and

customer-supplied parts.2. The operation-maintenance stage

Include specialize maintenance components as well as development life cycle components, which are applied mainly for functionality improving maintenance tasks.

62

Components of infrastructure error prevention and improvement :

Main objectives of these components, which are applied throughout the entire organization, are :

To eliminate or at least reduce the rate of errors, based on the organization’s accumlated SQA experience.

63

Components of software quality management :

This class of components is geared toward several goal:

The major ones being the control of development and maintenance activities and introduction of early managerial support actions that mainly prevent or minimize schedule and budget failures and their outcomes.

64

Components of standardization, certification, and SQA system assessment

The main objective of this class are:

1. Utilization of international professional knowledge

2. Improvement of coordination of the organizational quality system with other organizations

3. Assessment of the achievements of quality systems according to a common scale.

The various standards classified into 2 groupes: Quality management standards Project process standards.

65

Organizing for SQA- the human components

The SQA organizational base includes : Managers Testing personnel The SQA unit and practitioners interested in SQ.

The main objectives are to initiate and support the implementation of SQA

components Detect deviation from SQA procedures and

methodology Suggest improvements

66

Part II Pre-project SQ components Chapter 5

Contract Review

67

Contract Review

Is the software quality element that reduces the probability of undesirable situation like in

( CFV project ).

Contract review is a requirement by the ISO 9001 and ISO 9000-3 guidelines.

68

The Contract review process and its stages

Several situations can lead a SW company to sign a contract with a customer such as : Participation in a tender Submission of a proposal according to the

customer’s RFP. Receipt of an order from a company’s customer Receipt of an internal request or order from

another department in the organization

69

The Contract review process and its stages

Contract review : is the SQA component devised to guide review

drafts of proposal and contract documents.

If applicable, provides oversight ( supervision ) of the contracts carried out with potential project partners and subcontractors.

70

The Contract review process itself is conducted in two stages :

Stage 1 – Review of the proposal draft prior to submission to the potential customer ( proposal draft review ): Reviews the final proposal draft and proposal’s foundations: Customer’s requirement documents Customer’s additional details and explanations of the

requirements Cost and resources estimates Existing contracts or contract drafts of the supplier with

partners and subcontractors.

71

The Contract review process itself is conducted in two stages :

Stage 2 – Review of the proposal draft prior to signing ( Contract draft review ): Reviews the contract draft on the basis of the proposal and the understandings ( include changes ) reached during the contract negotiations sessions.

The individuals who perform the review thoroughly examine the draft while referring to a comprehensive range of review subjects ( a Check-list ) is very helpful for assuring the full coverage of relevant subjects.

See appendix 5A, 5B

72

Contract Review objectives:

Proposal draft review objectives( assure the following ) Customer requirements have been clarified and documented Alternative approaches for carrying out the project have been

examined Formal aspects of the relationship between the customer and SW

firm have been specified. Identification of development risks Adequate estimation of project resources and timetable have been

prepared. Examination of the customer’s capacity to fulfill his commitments Definition of partners and subcontractors participation conditions Definition and projection proprietary rights.

73

Contract Review objectives:

Contract draft review objectives( assure the following ) No un-clarified issues remain in the contract draft All the understandings reached between the customer

and the firm are to be fully and correctly documented. No changes, additions, or omissions that have not been

discussed and agreed upon should be introduced into contract draft.

74

Factors affecting the extent of a contract review:

Project magnitude, usually measured in man-month resources.

Project technical complexity Degree of staff acquaintance with and experience in

the project area. Project organizational complexity, the greater the

number of organizations ( partners, subcontractors, and customers ) taking part in the project, the greater the contract review efforts required.

75

Who performs a contract review:

The leader or another member of the proposal team The members of the proposal team An outside professional or a company staff member

who is not a member of the proposal team. A team of outside experts.

76

Implementation of a contract review of a major proposal

The characteristics of the major proposal : Very large-scale project Very high technical complexity New professional area for the company High organizational complexity

The difficulties of carrying out contract reviews for major proposals : Time pressures Proper contract review requires substantial professional work The potential contract review team members are very busy.

77

Implementation of a contract review of a major proposal

Recommended avenues ( approaches ) for implementing major contract reviews : The contract review should be scheduled. A team should carry out the contract review A contract team leader should be appointed The activities of the team leader include :

Recruitment of the team members Distribution of review tasks Coordination between members Coordination between the review team and the proposal team Follow-up of activities, especially compliance with the schedule Summarization of the findings and their delivery to the proposal team.

78

Contract review for internal projects

See table 5.1 page 86 The main point here is the internal relationship. Loose relationships are usually characterized by

insufficient examination of the project’s requirements, its resources and development risks.

To avoid the previous problems we have to apply the contract review to the internal as external projects by implementing procedures that define : An adequate proposal for the internal project Applying a proper contract review process An adequate agreement between the internal customer and the

internal supplier.

79

Chapter 6Development and quality plans

Development plans and quality plans are the major elements needed for project compliance with ISO 9000.3 standards and ISO/IEC 2001 and with IEEE 730.

It is also an important element in the Capability Maturity Model ( CMM ) for assessment of SW development organization maturity.

The projects needs development and quality plans that : Are based on proposal materials that have been re-examined and

thoroughly updated Are more comprehensive than the approved proposal, especially

with respect to schedules, resources, estimates, and development risk evaluations

Include additional subjects, absent from the approved proposal others

80

Development plan and quality plan objectives

1. Scheduling development activities that will lead to successful and timely completion of the project, and estimating the required manpower resources and budget.

2. Recruiting team members and allocating development resources.

3. Resolving development risks.4. Implementing required SQA activities5. Providing mgt. with data needed for project

control.

81

Elements of the development plan

1. Project products2. Project interfaces3. Project methodology and development tools4. SW development standards and procedures5. The mapping of the development process.( proj. mgt. Gant )6. Project milestones ( documents , code , report )7. Project staff organization ( org. stru., prof. req., no of team

mem., names of team leaders )8. Development facilities ( SW, HW tools, space, period req.

for each use )9. Development risks ( see next slide )10. Control methods11. Project cost estimation

82

Development risks

Is a state or property of a development task or environment which, if ignored, will increase the likelihood of project failure. Such as :

1. Technological gap

2. Staff shortages

3. Interdependence of organizational elements- the likelihood that suppliers or specialized HW or SW subcontractors, for example, will not fulfill their obligations or schedule.

83

Elements of Quality Plan

1. Quality goals ( quantitative measures example page 102 )2. Planned review activities

The scope of review activity The type The schedule ( priorities ) The specific procedure to be applied Who responsible for carrying out the rev. act.

3. Planned SW tests ( a complete list of planned SW tests should be provided ) each test

The unit, integration or the complete system to be tested The type of testing activities to be carried out The planned test schedule The specific procedure Who responsible

84

Elements of Quality Plan

4. Planned acceptance tests for externally developed SW

5. Configuration management configuration mgt tools and procedures, including those change-control procedures meant to be applied throughout the project

85

Dev. And Quality Plan for small projects & internal projects

See page 105 , 106

86

Chapter 7Integrating Quality activities in the project life cycle

Classic and Other SW development Methodologies: SDLC ( Req. def. , Analysis, Design, Coding, sys. Tests, install and

conversion, op. and maintenance )

87

Integrating Quality activities in the project life cycle

Prototyping

88


The Spiral model See page 128 It is an improved metho. for overseeing large and more complex

projects Combines SDLC & prototyping At each iteration of the spiral the following activities are

performed: Planning Risk analysis and reslution Engineering activities Customer evaluation, comm, changes, etc

89


The object-oriented model. Easy integration of existing sw modules ( Objects ) into newly

developed sw sys. A SW component library serves this purpose by supplying sw

components for reuse. See page 130 Advantages of library reuse:

Economy Improve quality Shorter development time

The advantages of OOPS will grow as the storage of reusable SW grows ( Example Microsoft and Unix )

90

Factors affecting intensity of quality assurance activities in the development projects

Quality assurance activities will be integrated into development plan that implements one or more SW development models

Quality assurance planners for project are required to determine : The list of QA activities needed for a project For each QA activity:

Timing Who perform & the resources required Team members, external body for QA

Resources required for removal of defects and introduction of changes.

91

Factors affecting intensity of quality assurance activities in the development projects

Project factors Magnitude of the project Technical complexity and difficulty Extent of reusable SW components Severity of failure outcome if the project fails

Team factors Professional qualification of team members Team acquaintance with the project and its experience in the area Availability of staff members who can professionally support team Familiarity with team members, in other words the percentage of

new staff members in the team

See example page 132

92

Verification, Validation and Qualification

Three aspects of quality assurance of the SW product are examined under the issues of verification, validation, and qualification( IEEE std 610.12-1990)

Verification : the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. It examines the consistency of the products being developed

with products developed in the previous phases. Examiner can assure that development phases have been

completed correctly

93


Validation : the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. It represents the customer’s interest by examining the extent

of compliance to his original req. Comprehensive validation reviews tend to improve customer

satisfaction from the system

94


Qualification : the process used to determine whether a system or component is suitable for operational use. It focuses on operational aspects, where maintenance is the

main issues

Planners are required to determine which of these aspects should be examined in each quality assurance activity.

95

A model for SQA defect removal effectiveness & Cost

The model deals with 2 quantitative aspects :

1. Effectiveness in removing project defects

2. The cost of removal See page 135

96

Defect removal effectiveness

It is assumed that any SQA activity filters ( screens ) a certain percentage of existing defects.

In most cases the percentage of removed defects is somewhat lower than the percentage of detected defects as some corrections are ineffective or inadequate.

The next SQA activity will faces both the remaining defects and the new defects created in the current development phases.

It is assumed that the filtering effectiveness of accumulated defects of each QA activity is not less than 40%.

Table 7.4 page 136 list the average filtering effectiveness by QA activities.

97

Cost of defect removal

The cost of defect removals varies by development phase, while costs rise substantially as the development process proceeds.

Example : removal of a design defect detected in the design phase may require an investment of 2.5 working days; removal of the same defect may required 40 days during the acceptance tests.

Defect-removal costs based on some surveys are shown in table 7.5 page 137.

98

The Model The model is based on the following assumptions:

The development process is linear and sequential, following the waterfall model.

A number of new defects are introduced in each development phase ( see table 7.3 page 135 ).

Review and test SQA activities serve as filters, removing a percentage of the entering defects and letting the rest pass to the next phase. If we have 30 defects and the filtering efficiency 60% then 18 defects will be removed & 12 will stay to the next.

At each phase the incoming defects are the sum of defects not removed together with the new defects introduced ( created ) in the current development phase.

The cost is calculated for each QA activity by multiplying the number of defects removed by the relative cost of removing a defect. ( table 7.5 )

The remaining defects passed to the customer, will be detected by him.

99

The model presents the following

POD : phase originated defects ( table 7.3 ) PD : passed defects. %FE : % filtering effectiveness ( table 7.4 ) RD : removed defects CDR : cost of defect removal ( table 7.5 ) TRC : total removal cost.

100

Chapter 8 Reviews

IEEE definition Review process :

A process or meeting during which a work product or set of products is presented to project personnel , managers, users, customers, or other interested parties for comment or approval.

101

Methodologies for reviewing documents

Reviews acquire special importance in the SQA process because they provide early direction and prevent the passing of design and analysis errors “down-stream” , to stages where error detection and correction are much complicated and costly :

The methodologies for reviewing : Formal design review Peer reviews ( inspections and walkthroughs ) Expert opinions

Standards for SW reviews are the subject of IEEE std 1028 ( IEEE, 1997 ).

102

Reviews Objectives ( Direct Objectives )

To detect analysis & design errors as well as subjects where corrections, changes and completions are required with respect to the original specifications and approved changes.

To identify new risks likely to affect completion of the project.

To locate deviations from templates and style procedures and conventions. Correction of these deviations is expected to contribute to improved communication & coordination resulting from greater uniformity of methods & documentation style.

To approve the analysis or design product. Approval allows the team to continue to the next development phase.

103

Reviews Objectives ( Indirect Objectives )

To provide an informal meeting place for exchange of professional knowledge about development methods, tools, and techniques.

To record analysis and design errors that will serve as a basis for future corrective actions. The corrective actions are expected to improve development methods by increasing effective and quality, among other product features.

104

Formal design reviews ( DRs )

Formal design review, also called Design reviews ( DRs ) Formal technical reviews ( FTR )

Without this approval, the development team cannot continue to the next phase of SW development project.

Formal design review can be conducted at any development milestone requiring completion of an analysis or design document, whether that document is a requirement specification or an installation plan.

105

A list of common Formal design reviews :

DPR - development plan review SRSR- Software requirement specification review PDR – Preliminary design review DDR – Detail design review DBDR – Data base design review TPR – Test plan review STPR – Software test procedure review VDR – Version description review OMR – operator manual review SMR – Support manual review TRR – Test readiness review PRR – Product release review IPR – Installation Plan review

106

The Formal Design Review will focus on :

The participants The prior preparations The DR session The recommended post-DR activities

107

The participants in a DR

All DRs are conducted by A review leader A review team

The review leader: characteristics Knowledge & experience in development of projects of the

type reviewed. Seniority at a level similar to if not higher than that of the

project leader A good relationship with the project leader and his team A position external to the project team.

Small dev. Departments and software houses typically have difficulties finding an appropriate candidate to lead the review team. One possible solution to this is the appointement of an external consultant.

108

The Review Team

It is desirable for non-project staff to make up the majority of the review team.

The size of the review team from 3 to 5 to be an efficient team

109

Preparation for a DR

A DR session are to be completed by all three main participants in the review : Review leader , an team Development team.

Each one is required to focus on distinct aspects of the process.

Review leader preparations (main tasks) : To appoint the team members To schedule the review sessions To distribute the design document among team members ( hard

copy, electronic copy etc )

110

Preparation for a DR

Review team preparations (main tasks) : Review the design document and list their comments

prior to the review session team members may use a review checklists. See chapter 15 ( checklists )

Development team preparations ( main tasks ) Prepare a short presentation of the design document The presentation should focus on the main

professional issues awaiting approval rather than wasting time on description of the project in general.

111

The DR session

The agenda is the issue ( a typical DR session agenda ) :

1. A short presentation of the design document2. Comments made by members of the review team.3. Verification and validation in which each of the

comments is discussed to determine the required actions ( corrections, changes and addition ) that the project team has to perform.

4. Decisions about the design product ( document ), which determines the project’s progress. These decisions take the following three forms:

112

Decisions forms :

Full approval : enables immediate continuation to the next phase. It may be accompanied by demands for some minor corrections to be performed by project team.

Partial approval: approval of immediate continuation to the next phase for some parts of the project, with major action items demanded for the remainder of the project.

Denial of approval : demands to repeat of the DR

113

The DR report see appendix 8A

one of the review leader responsibilities is to issue a DR report immediately after the review session.

The development team should perform the corrections earlier and minimize the attendant delays to the project schedule.

The report major sections contain : A summary of the review discussion The decision of the continuation of the project A full list of the required actions ( corr, changes, additions)

and the anticipated completion dates. The name(s) of the review team member(s) assigned to

follow up performance of corrections.

114

The follow-up process

The review leader himself is required to determine whether each action item has been satisfactory accomplished as a condition for allowing the project to continue to the next phase.

Follow-up should be documented to enable clarification.

115

Pressman (2000, chapter 8 )

Pressman’s 13 “golden guidelines “ for a successful design review:

See page 157

116

Peer Reviewstwo review methods ( Inspection and Walkthrough )

The major difference between formal design reviews and peer review methods is rooted in participants & authority.

In peer reviews, as expected, the project leader’s equals, members of his department and other units.

The other difference lies in degree of authority & the objective of each review method.

The peer review main objectives lies in detecting errors & deviations from standards.

The appearance of the CASE tools reduce the value of manual reviews such as inspection and walkthrough.

Researches find out that peer reviews are highly efficient as well as effective method.

117

Inspection & Walkthrough

What differentiates a walkthrough from an inspection is the level of formality, inspection is the more formal of two.

Inspection emphasizes the objective of corrective actions.

Walkthrough’s findings are limited to comments on the document reviewed.

118

Inspection & Walkthrough

Inspection is usually based on a comprehensive infrastructure, including : Development of inspection checklists developed for each

type of design document as well as coding language and tool, which are periodically updated.

Development of typical defect type frequency tables, based past findings, to direct inspectors to potential “defect concentration areas”.

Periodic analysis of the effectiveness of past inspections to improve the inspection methodology

Introduction of scheduled inspections into the project activity plan and allocation of the required resources, including resources for correction of detected defects.

119

Participants of peer reviews

A review leader Main tasks & qualification page 161

The author Invariably a participant in each type of peer review.

Specialized professional For inspections:

A designer A coder or implementer A tester

For walkthrough: A standards enforcer A maintenance expert A user representative.

120

Team assignments

The presenter The Scribe

121

Preparations for a peer review session

Leader preparation Team’s preparation

122

Session Documentation

Inspection session findings report Prepared by the scribe

Inspection session summary report Prepared by the leader

See appendix 8b , 8c

123

Post-Peer review activities

Post-inspection activities are conducted to attest to : The prompt, effective correction and reworking of all

errors by the designer/author and his team, as performed by the inspection leader in the course of the assigned follow-up activities.

Transmission of the inspection reports to the internal Corrective Action Board ( CAB ) for analysis.

See Fig 8.2 ( comparison of the peer review methods ( Page 166 )

124

The efficiency of peer reviews

Some of the more common metrics applied to estimate the efficiency of peer reviews: peer review detection efficiency( average hrs

worked per defect detected) Peer review defect detection density ( average

number of defects detected per page of the design document )

Internal peer review effectiveness ( % of defects detected by peer reviews as % of total defects detected by the developer).

125

Comparisons

See tables page 167-169

126

Expert opinions ( external )

It is good in the following situations : Insufficient in-house proff. Temporary lack in-house proff. Disagreements In small organizations

127

Chapter 9Software testing - Strategies

Testing Definition : Testing is the process of executing a program with

intention of finding errors. IEEEdefinition:

1. The process of operating a system or component under specified condition, observing or recording the results, and making an evaluation of some aspect of the system or component.

2. The process of analyzing a software item to detect the difference between the existing and required conditions ( that is, bugs ) and evaluate the features of the software item.

128

Software testing - Definition

Is a formal ( SW test plan ) process carried out by specialized testing team ( independent ) in which a software unit, several integrated software units or entire software package are examined by running the programs on a computer. All the associated tests are performed according to approved test procedures on approved test case.

129

Software testing objectives

Direct objectives: To identify and reveal as many errors as possible in

the tested SW. To bring the tested SW, after correction of the

identified errors and retesting, to an acceptable level of quality.

To perform the required tests efficiently, within budgetary and scheduling limitations.

Indirect objectives To compile a record of SW errors for use in error

prevention ( by corrective & preventive actions )

130

Software testing Strategies

To test the SW in its entirety, once the completed package is available; otherwise known as “big bang testing “.

To test the SW piecemeal, in modules, as they are completed ( unit tests ) ; then to test groups of tested modules integrated with newly completed modules ( integrated tests ). This process continues until all the entire package is tested as whole ( system test ). This testing strategy is usually termed “incremental testing”

131

Incremental testing is also performed according to two basic Strategies:

Bottom-Up ( 4 stages ) see fig page 183 Top-down ( 6 stages ) The incremental pathes:

Horizontal sequence ( breadth first ) Vertical sequence ( Depth first )

132

Stubs & Drivers for incremental testing

Stubs and drivers are SW replacement simulators required for modules not available when performing a unit test or an integration test.

Stubs ( often termed a “Dummy Module “ )replaces an unavailable lower level module, subordinate to the module tested.

It is required for top-down testing of incomplete systems. See example fig 9.2

133

Stubs & Drivers for incremental testing

A driver is a substitute module but of the upper level module that activates the module tested.

The driver is passing the test data on to the tested module and accepting the results calculated by it.

It is required in bottom-up testing until the upper level modules are developed.

See example fig 9.2

134

Bottom-Up Vs Top-Down strategies

Main Adv. Of bottom-up : The relative ease of its performance

Main disadv. The lateness at which the prog. As whole can be observed ( at the stage

following testing of the last module ) Main adv. Of Top-down:

The possibility it offers to demonstrate the entire prog. Function shortly after activation of the upper-level modules has been completed. This characteristic allows for early identification of analysis & design errors related to algorithms, functional requir. , and the like.

Main disadv. The relative difficulty of preparing the required stubs, with often require

very complicated programming. The relative difficulty of analyzing the result of the tests.

135

Big bang Vs. Incremental testing

The main disadvantages of big bang : Identification of errors becomes difficult Corrections will be at the same time. Error correction estimation of the required testing

resources and testing schedule a rather fuzzy endeaver. Incremental testing adv.:

Usually performed on relatively small SW modules, as unit or integration tests.( more errors detection )

Identification and correction of errors is much simpler and requires fewer resources because it is performed on a limited volume of SW.

136

Software test classification.Classification according to testing concept:

Two testing classes have been developed: Black box ( functionality ) testing:

Identifies bugs only according to SW malfunctioning as they are revealed in its erroneous output.

Incases that outputs are found to be correct, black box testing disregarded the internal path of calculations and processing performed.

White box ( structural ) testing: Examines internal calculation paths in order to identify bugs. The white is meant to emphasize the contrast between

methods

137

Software test classification.Classification according to requirements:

See table 9.1 Page 188

138

White Box Testing

White box testing concept requires verification of every program statement and comment.

See table 9.2 , white box testing enables performance of data processing and calculations correctness tests SW qualification tests Maintainability tests Reusability tests

Every computational operation in the sequence of operations created by each test case ( “Path” ) must be examined.

This type of verification allows us to decide whether the processing operations and their sequences were programmed correctly for the path in question, but not for other pathes.

139

Data processing and calculation correctness testsPath coverage & line coverage

Path coverage : total number of possible paths 10 if-then-else : 1024 paths

Line coverage: for full line coverage, every line of code be executed at least once during the process of testing. The line coverage metrics for completeness of a line-testing plan are defined as the percentage of lines indeed executed during the tests. Flow chart and a program flow graph are used.

Example 191.

140

McCabe’s cyclomatic complexity metrics

Measures the complexity of a program or module at the same time as it determines the maximum number of independent paths needed to achieve full line Coverage of the program.

The measure is based on graph theory using program flow graph.

An independent path is defined with reference to the succession of independent paths accumulated.

Independent path is any path on the program flow graph that includes at least one edge that is not included in any former independent paths.

See table 9.5 page 194

141

McCabe’s cyclomatic complexity metrics

The cyclomatic complexity metric V(G) V(G) = R = E – N + 2 = P + 1 Where R = the number of regions E = number of edges N = number of nodes P = number of decisions ( nodes having more than one leaving edge). See example page 195 An empirical studies show that if V(G) < 5 considered simple If it is 10 or less considered not too difficult If 20 or more it is high If it is exceed 50 the SW for practical purposes becomes untestable.

142

SW qualification & reusability testing

SW qualification testing SW reusability testing Adv & disadv. Of White box Page 197

143

Date post:	16-Jan-2016
Category:	Documents
Upload:	sidone
View:	18 times
Download:	0 times

Chapter 1

Documents