Date post: | 12-Nov-2014 |
Category: |
Documents |
Upload: | softwarecentral |
View: | 2,361 times |
Download: | 0 times |
1 / 28
CS 425/625 Software Engineering
Verification and Validation
Based on Chapter 19 of the textbook [SE-6] Ian Sommerville,Software Engineering, 6th Ed., Addison-Wesley, 2000 and on theCh19 PowerPoint presentation available at the book’s web-site:www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html
November 10, 2003
2 / 28
Outline
Introduction Testing and Debugging Verification and Validation Planning Software Inspections Program Inspections (forms of Software
Inspections) Automatic Static Analysis
3 / 28
Introduction…..
Verification and validation (V&V): the checking and analysis processes that ensure the software satisfies its specification and meets the needs of the clients who are paying for it
Validation: “Are we building the right product?” Verification: “Are we building the product right?” Verification involves checking the software conforms
with its specification while the more general process of validation ensures the software meets the needs of the clients
V&V is a whole life-cycle process, encompassing requirements reviews, design reviews, code inspections, and program testing
4 / 28
.Introduction.…
V & V techniques:V & V techniques: Software inspectionsSoftware inspections Software testingSoftware testing
Software inspections Are static V&V techniques as they do not require the Are static V&V techniques as they do not require the
software to be executed software to be executed Consist of inspections, automated static analyses, and formal Consist of inspections, automated static analyses, and formal
verifications of source code or system modelsverifications of source code or system models Can only check the correspondence between the software
and its specification Cannot demonstrate the system is operationally useful Cannot check non-functional requirements of the software
5 / 28
..Introduction...
Software testing It is a dynamic V&V technique as it needs an
executable version of the software system The system is executed with test data and its
operational behaviour is assessed Can reveal the presence of errors, not their absence A successful test is a test which discovers one or
more errors The V&V technique for checking non-functional
requirements and for verifying system integration Should be used in conjunction with static techniques
to provide full V&V coverage
6 / 28
…Introduction..
Types of software testing Defect testing
Intended to find inconsistencies between a program and its specification
Tests designed to discover program faults and defects A successful defect test is one which reveals the presence
of defects in a system
Statistical testing Designed for software’s performance and reliability
estimation By running tests that reflect actual user inputs and their
frequency, an estimate of operational reliability can be made
7 / 28
….Introduction.
Static and dynamic V&V [Fig 19.1, Somm00]
Formalspecification
High-leveldesign
Requirementsspecification
Detaileddesign
Program
PrototypeDynamicvalidation
Staticverification
8 / 28
…..Introduction
Verification and validation should establish confidence that the software is fit for purpose
This does not mean the software is completely free of defects; rather, it must be good enough for its intended use
The required level of confidence depends on: Software’s purpose: the level of confidence depends on
how critical the software is to an organisation User expectations: users may have lower expectations
of certain types of software Marketing environment: getting a product to market
early may have higher priority than finding its defects
9 / 28
Testing and debugging.
Defect testing and debugging are distinct processes
V&V is concerned with establishing the existence of defects in a program
Debugging is concerned with locating and repairing these errors
Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the errors
10 / 28
.Testing and debugging
The debugging process [Fig. 19.2, Somm00]
Locateerror
Designerror repair
Repairerror
Re-testprogram
Testresults Specification Test
cases
11 / 28
The planning of V&V should start early in the development process
The plan should balance static verification and testing, specify testing standards and procedures, establish checklists for inspections, and define the software test plan
Test planning breaks down V&V into a number of stages, often organized according to the V-model (shown on next page)
The focus is on setting standards and procedures for inspections and testing, not on describing product tests
V & V Planning..
12 / 28
.V&V Planning..V&V Planning.
Requirementsspecification
Systemspecification
Systemdesign
Detaileddesign
Module andunit codeand tess
Sub-systemintegrationtest plan
Systemintegrationtest plan
Acceptancetest plan
ServiceAcceptance
testSystem
integration testSub-system
integration test
The V-model of development [Fig. 19.3, Somm00]
13 / 28
..V&V Planning..V&V Planning
The structure of a software test plan The testing process Requirements traceability Tested items Testing schedule Test recording procedures Hardware and software requirements Constraints
14 / 28
Software Inspections.
Involve people examining software code or models (representations) with the aim of discovering defects
Do not require execution of a system thus can be used throughout the development process
May be applied to any representation of the system: requirements, design, test data, etc.
Very effective technique for discovering errors
15 / 28
.Software Inspections
In software inspections many different defects can be discovered in a single review of the source code or software model
In testing, one defect may mask another hence several executions are required
Software inspections reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly occur
Software inspections and software testing are complementary, not competing techniques (see also slides 4 and 5)
16 / 28
Program Inspections…….
Program inspections are a type of software inspections
Consist of formal reviews conducted by teams and intended for program defect detection
Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g., an un-initialized variable), or non-compliances with standards
17 / 28
.Program Inspections……
Inspection pre-conditions: A precise specification must be available Team members must be familiar with the
organisation standards Syntactically correct code must be available An error checklist should be prepared Management must accept that inspection will
increase costs early in the software process Management must not use inspections for staff
appraisal
18 / 28
Inspectionmeeting
Individualpreparation
Overview
Planning
Rework
Follow-up
..Program Inspections…..
The inspection process [Fig. 19.6, Somm00]
19 / 28
…Program Inspections….
Program inspection procedure: The inspection is planned by the moderator (or chairman) The system overview is presented to the inspection team by the
program author (or owner) The code and associated documents are distributed to the
inspectors (inspection team) in advance for individual preparation
The inspection meeting takes place and errors are noted (the inspection also involves a reader and, possibly, a scribe)
During re-work modifications are made by the author to repair discovered errors
Re-inspection may or may not be required, based on moderator’s decision
20 / 28
….Program Inspections…
Inspection teams are Inspection teams are made up of at least 4 members: Author of the code being inspected Inspector who finds errors, omissions and
inconsistencies Reader who reads the code to the team Moderator who chairs the meeting and notes
discovered errors Other roles are chief moderator and scribe
21 / 28
…..Program Inspections..
Inspection checklist A checklist of common errors should be used
during the inspection This error checklist is programming language
dependent The weaker the type-checking of the language,
the larger the checklist Examples of possible errors: initialisation,
constant naming, loop termination, array bounds, etc.
22 / 28
Inspection checks (fault classes) [Fig. 19.7, Somm00]
Fault class Inspection checkData faults Are all program variables initialised before their values
are used?Have all constants been named?Should the lower bound of arrays be 0, 1, or somethingelse? Should the upper bound of arrays be equal to the size ofthe array or Size -1?If character strings are used, is a delimiter explicitlyassigned?
Control faults For each conditional statement, is the condition correct?Is each loop certain to terminate?Are compound statements correctly bracketed?In case statements, are all possible cases accounted for?
Input/output faults Are all input variables used?Are all output variables assigned a value before they areoutput?
Interface faults Do all function and procedure calls have the correctnumber of parameters?Do formal and actual parameter types match? Are the parameters in the right order? If components access shared memory, do they have thesame model of the shared memory structure?
Storage managementfaults
If a linked structure is modified, have all links beencorrectly reassigned?If dynamic storage is used, has space been allocatedcorrectly?Is space explicitly de-allocated after it is no longerrequired?
Exceptionmanagement faults
Have all possible error conditions been taken intoaccount?
23 / 28
………….Program Inspections.Program Inspections
Inspection rates: 500 statements/hour during overview 125 source statement/hour during individual
preparation 90-125 statements/hour can be inspected Inspection is therefore an expensive process Inspecting 500 lines costs about 40 man/hours
effort = £2800
24 / 28
Automated Static Analysis….
Static analyzers are software tools for source code processing
They parse the program text to discover erroneous conditions
Very effective as an aid to inspections; supplement but cannot replace inspections
Particularly valuable for languages such as C that have weak typing (many errors can remain undetected by the compiler)
Less cost-effective for languages such as Java that have strong type checking (many errors can be detected during compilation)
25 / 28
.Automated Static Analysis…
Fault class Static analysis checkData faults Variables used before initialisation
Variables declared but never usedVariables assigned twice but never usedbetween assignmentsPossible array bound violations Undeclared variables
Control faults Unreachable codeUnconditional branches into loops
Input/output faults Variables output twice with no interveningassignment
Interface faults Parameter type mismatchesParameter number mismatchesNon-usage of the results of functionsUncalled functions and procedures
Storage managementfaults
Unassigned pointersPointer arithmetic
Automatic static analysis checks [Fig. 19.8, Somm00]
26 / 28
..Automated Static Analysis..
Stages of static analysis: Control flow analysis. Checks for loops with
multiple exit or entry points, finds unreachable code, etc.
Data use analysis. Detects un-initialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc.
Interface analysis. Checks the consistency of routine and procedure declarations and their use
27 / 28
…Automated Static Analysis.
Stages of static analysis [cont’d]:
Information flow analysis. Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code review (inspection)
Path analysis. Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process
28 / 28
LINT static analysis
[Fig 19.9, Somm00]
138% more lint_ex.c
#include <stdio.h>printarray (Anarray) int Anarray;{ printf(“%d”,Anarray);}main (){ int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ;}
139% cc lint_ex.c140% lint lint_ex.c
lint_ex.c(10): warning: c may be used before setlint_ex.c(10): warning: i may be used before setprintarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10)printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10)printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11)printf returns value which is always ignored