Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | beverley-margaret-black |
View: | 214 times |
Download: | 0 times |
James Nowotarski
10 October 2006
SE 325/425Principles and
Practices of Software Engineering
Autumn 2006
2
Topic Duration
Requirements process recap 30 minutes
V model 45 minutes
Testing techniques 15 minutes
*** Break
Current event reports 30 minutes
Testing techniques 60 minutes
Wrap-up 15 minutes
Today’s Agenda
3
Context
Communication project initiation requirements
Modeling analysis design
Construction code test
Deployment delivery support
Planning & Managing
elicitationRequirementsengineeringtasks (Ch. 7-8)
elaborationspecification
Primarydeliverables
functional reqtsnon-functional reqts
analysis modelsoftware reqts spec
Begin with the end in mind - Sample SRSOverviewRevision HistoryTable of Contents1.0 Introduction
1.1 Purpose1.2 Scope1.3 References1.4 Assumptions and Dependencies
2.0 Use-Cases3.0 Requirements
3.1 Functional Requirements3.2 Non-Functional Requirements
3.2.1 Usability3.2.2 Reliability3.2.3 Performance3.2.4 Supportability
4.0 Online User Documentation and Help System
Requirements5.0 Design Constraints6.0 Purchased Components7.0 Interfaces
7.1 User Interfaces7.2 Hardware
Interfaces7.3 Software Interfaces7.4 Communication
Interfaces8.0 Licensing Requirements9.0 Legal, Copyright, and Other
notices10.0 Applicable StandardsIndexGlossary
5
Functional vs. Non-Functional
A functional requirement (FR) describes what the system needs to do.
Example: ‘The system shall display the current customer balance’.
6
Functional vs. Non-Functional
A non-functional requirement (NFR) describes a constraint upon the solution space.
Examples: Performance, flexibility, reliability, usability, portability, maintainability, safety, and security.Also called “quality” requirements, “ilities”, or even “systemic” requirements. Emergent Properties: An NFR that is realized through the careful implementation of other requirements on which it depends. Example: “The query must return its results in less than three seconds” is only realizable once the architecture and much of the system functionality has been implemented.
7
The Requirements Process
Elicitation: Proactively working with stakeholders to discover their needs, identify potential conflicts, and establish a clear scope and boundaries for the project.
Elaboration (Analysis): Gaining a deeper understanding of the product and its interactions.
Specification: Production of a series of documents that capture the system and software requirements in order to support their systematic review, evaluation, and approval.
Validation: Inspecting requirements to ensure their correctness.
Management: Issues such as software configuration management, traceability, impact analysis, and version control.
8
Key Question: Deliverables
Steps Techniques
What does the system need to do?How well does it need to do it?
Functional requirementsQuality requirements
1. Review as-is system2. Identify requirements of
to-be system
Re-engineering AHPInterviewing PrototypingObservationSurveys/Focus GroupsJoint Application Design (JAD)Benchmarking
Elicitation
Roles Estimating guidelines
Business analyst
9
Joint Application Design (JAD)
10
Elicitation Techniques: AHP
Develop Software
Performance Usability Flexibility
ArchitectureChoice 1
ArchitectureChoice 2
Goal
Alternatives
Qualityreqts
.08 .64 .28
.41.59
11
Requirement Qualities
Each individual requirement should be:
Concise
Correct
Non-ambiguous
Feasible
Verifiable
Traceable
Manageable
12
Topic Duration
Requirements process recap 30 minutes
V model 45 minutes
Testing techniques 15 minutes
*** Break
Current event reports 30 minutes
Testing techniques 60 minutes
Wrap-up 15 minutes
Today’s Agenda
13
Where time is spent on big systems projects
Activity % of time
Planning and Modeling
33%
??? 50%
All other Construction and Deployment
17%
14
Why we test
We test software because we cannot guarantee its correctness – under normal development practices.
Testing is the art of devising and executing test cases that have a high likelihood of finding errors.
A small subset of faults accounts for most failures during operation. We need to ‘test smart’ in order to find these
faults.
15
Verification & Validation
Testing is just part of a broader topic referred to as Verification and Validation (V&V)
Pressman/Boehm: Verification: Are we building the product right? Validation: Are we building the right product?
IEEE standard 1012-1998: Requirements validation is the process of
evaluating an implemented system to determine whether it conforms to the specified requirements.
16
Some overriding principles
DRTFTDo it right the first time
“The first mistake that people make is thinking that the testing team is responsible for assuring quality.”-- Brian Marick.
Stage Containment
17
Stage Containment
Communication project initiation requirements
Modeling analysis design
Construction code test
Deployment delivery support
Planning & Managing
errorError detection defect fault
Error origination
18
V-Model
Requirements
Functional Design
IntegrationTest
Unit Test
Code
TechnicalDesign
DetailedDesign
AcceptanceTest
SystemTest
Testing: Test that the product implements the specification
Flow of Work
Verification
Validation
Legend:
19
Terminology
Testing - Ensures that the components of the application are put together correctly, according to the different levels ofspecification. Testing consists of exercising a newly integrated portion of the application by running a number of test cases in controlled mode. Each test case is designed to test a statement of the specification.
Verification - Verification is the checking of a deliverable against a standard of work set out for the process that produces the deliverable. The sources for this standard include: 1) Exit criteria defined for each task; 2) Design and coding standards set up by the project team; 3) Evidence that the process prescribed for executing the activity has been followed; 4) Consistency and completeness checks; 5) Models and templates described as part of the application architecture; 6) Standards set up for using the technical architecture. Verification is usually done through reviews, inspections, and walk-throughs.
Validation - Validation is the checking of a deliverable against the specification/requirements implemented by that deliverable (e.g., checking a database design against the data model).
Stage Containment - Stage containment is a project management objective driven by the desire to minimize the number of defects or faults discovered after the work has been completed and handed off to the next stage of the development process. Activities to achieve stage containment include verification, validation, and testing.
Entry/Exit Criteria - Entry and exit criteria are predefined standards that deliverables must meet before exiting one development stage and entering another. A team handing work off to another part of the project must fully satisfy their exit criteria, while the receiving team verifies that the work meets their standard entry criteria. The entry criteria for a receiving team are frequently the same as the exit criteria for the delivering team.
V Model - The V model of verification, validation, and testing provides a structured testing framework throughout the development process and ensures that both verification and validation are applied to deliverables within a system.
20
Requirements
Functional Design
IntegrationTest
Unit Test
Code
TechnicalDesign
DetailedDesign
AcceptanceTest
SystemTest
Testing: Test that the product implements the specification
Flow of Work
Verification
Validation
Legend:
May include:• Operational
readiness test• Benefits
realization test
May include:• Performance test• Usability test• Stress test• Inter-Application
integration test• Hardware/Software
integration test• Restart/Recovery
test
V-Model: Optional Tests
21
Requirements
Functional Design
IntegrationTest
Unit Test
Code
TechnicalDesign
DetailedDesign
AcceptanceTest
SystemTest
Testing: Test that the product implements the specification
Flow of Work
Verification
Validation
Legend:
V-Model: The Testing Process
For each activity on left side of V:• Develop test conditions and
cycles• Develop entry and exit criteria
for corresponding test
For each activity on right side of V:• Set up environment and test team• Execute test and capture actual
and expected results, identify errors, correct, and regression test
22
Good tests
High probability of finding error Not redundant “Bang for buck”
23
Statement coverage:Goal is to execute each statement at least once.
Branch coverageGoal is to execute each branch at least once.
Path coverageWhere a path is a feasible sequence of statements that can be taken during the execution of the program.
What % of each type of coverage does this test execution provide?
5/10 = 50%
2/6 33%
¼ 25% Where does the 4 come from?
= tested
= not tested
Test Coverage Metrics
24
Design for testability
Understandability/Simplicity Operability/Controllability Antibugging
25
26
V-Model
Requirements
Functional Design
IntegrationTest
Unit Test
Code
TechnicalDesign
DetailedDesign
AcceptanceTest
SystemTest
Testing: Test that the product implements the specification
Flow of Work
Verification
Validation
Legend:
Unit testing
27
Unit testing Focuses on a single software component or
module. Design description guides test generation to
Ensure coverage of important control paths Test the boundaries of the module.
Focuses on internal processing logic and data structures.
Specific tests/Common errors
28
As no unit operates in a vacuum it is necessary to create stubs and drivers.
Moduleto betested
Stub Stub
Driver
RESULTS
Testcases
InterfaceLocal data structuresBoundary conditionsIndependent pathsError handling paths
Pressman: 6th ed., Figure 13.4
Unit test environment
29
V-Model
Requirements
Functional Design
IntegrationTest
Unit Test
Code
TechnicalDesign
DetailedDesign
AcceptanceTest
SystemTest
Testing: Test that the product implements the specification
Flow of Work
Verification
Validation
Legend:
Integration testing
30
Integration testing Integration testing is a systematic technique
for constructing the software architecture while at the same time conducting tests to uncover errors associated with the interfacing.
BIG BANG integration is not advisable! Incremental, piece-meal approaches:
Top-down Bottom-up Sandwich
Regression testing Ensure changes do not introduce unintended side
effects
31
Modules are integratedby moving downward throughthe control hierarchy.
Depth-first approachincorporates all components on a major control path.Example: M1, M2, M6, M8
Breadth-first approachincorporates allcomponents directly subordinate at each level.Example: M2, M3, M4.
M1
M3M2 M4
M6M5 M7
M8
Top-down integration
32
Ma
Mc
D3
Cluster1
Cluster3
Mb
Cluster 2
D1 D2
Bottom-up integration
33
The re-execution of some subset of tests that have already been conducted to ensure that changes have not introduced unintended side effects.
Whenever change is introduced or existing tests uncover errors that are fixed – there is opportunity for new errors to be introduced.
Supported by capture/playback tools. Regression testing includes:
A representative sample of tests that will exercise all software functions.
Additional tests that focus on software functions that are likely to be affected by the change.
Tests focusing on the software components that have been changed.
Regression testing
34
Topic Duration
Requirements process recap 30 minutes
V model 45 minutes
Testing techniques 15 minutes
*** Break
Current event reports 30 minutes
Testing techniques 60 minutes
Wrap-up 15 minutes
Today’s Agenda
35
White box vs. Black box
36
White box vs. Black box
Inputs designed to test a system function
Outputs (should match intended functionality)
37
White box vs. Black box
White box Black box
• Structural testing • Logic paths• Loops• Internal variables• Error conditions
• Functional testing • Incorrect/missing functions• Interface errors• Performance errors
38
V-Model
IntegrationTest
Unit Test
Code
AcceptanceTest
SystemTest
White box
Black box
39
First proposed by Tom McCabe in 1976.
Enables the test case designer to derive a logical complexity measure of the procedural design.
Uses this measure as the basis for defining an upper bound on the number of execution paths needed to guarantee that every statement in the program ins executed at least once.
Uses a notation known as a flow graph.
Each structured notation has a corresponding flow graph symbol.
Basis Path Testing
40
Sequence
if Case
while until
Where each circle represents one or more nonbranching set of source code statements.
Flow Graph Notation
41
Flow chart and corresponding flow graph.
1
2
3
6 4
57 8
910
11
1
2,3
10
11
6
9
8 4,57
42
Compound logic A compound condition occurs when one or more
Boolean operators (logical OR, AND, NAND, NOR) is present in a conditional statement.
Example:
if a OR bthen do Xelse do Yend if
a
b
xy
x
a
b
xy
43
Any path through the program that introduces at least one new set of processing statements or a new condition.
In terms of a flow graph, an independent path must move along at least one edge that has not previously been traversed.
In the previous example:path 1: 1-11path 2: 1-2-3-4-5-10-1-11path 3: 1-2-3-6-8-9-10-1-11path 4: 1-2-3-6-7-9-10-1-11
The path: 1-2-3-4-5-10-1-2-3-6-8-9-10-1-11 is NOT an independent path because it does not traverse any new edges.
Independence path
44
These paths constitute a basis set for the flow graph.
Design tests to execute these paths.
Guarantees: Every statement has been executed at least once. Every condition has been executed on both its true and false sides.
There is more than ONE correct set of basis paths for a given problem.
How many paths should we look for? Calculate cyclomatic complexity V(G) V(G) = E-N+2 (E = # edges, N = # nodes) V(G) = P + 1 (Where P = number of predicate nodes) V(G) = R (Where R = number of regions)
Basis Set
45
An Example:Procedure Average (Pressman, p.397)
* This procedure computes the average of 100 or fewer numbers that lie between bounding values; it also computes the sum of the total number valid.
INTERFACE RETURNS average, total.input, total.valid; INTERFACE ACCEPTS value, minimum, maximum;TYPE value[1:100] IS SCALAR ARRAY;TYPE average, total.input, total.valid;
minimum, maximum, sum IS SCALAR;TYPE i IS INTEGER;
46
Continued…
i = 1;total.input = total.valid = 0;sum = 0;DO WHILE value[i] <> -999 AND total.input < 100
increment total.input by 1;IF value[i] >= minimum and value[i] <= maximum
THEN increment total.valid by 1;sum = sum + value[i]
ELSE skipENDIFincrement I by 1;
ENDDO
IF total.valid > 0THEN average = sum / total.valid;ELSE average = -999;
ENDIF
END average
12
34
56
7
8
9
1011
1213
47
1. Use the design or code as a foundation and draw corresponding flow graph.
1
2
3
4
5
6
78
9
10
13
12 11
Steps for deriving test cases
2. Determine the cyclomatic complexity of the resultant flow graph.
V(G) = 17 edges – 13 nodes + 2 = 6V(G) = 5 predicate nodes + 1 = 6.
48
3. Determine a basis set of linearly independent paths.
1
2
3
4
5
6
78
9
10
13
12 11
Steps for deriving test cases
Path 1: 1-2-10-11-13Path 2: 1-2-10-12-13Path 3: 1-2-3-10-11-13Path 4: 1-2-3-4-5-8-9-2…Path 5: 1-2-3-4-5-6-8-9-2…Path 6: 1-2-3-4-5-6-7-8-9-2…
4. Prepare test cases that will force execution of each path in the basis set.
50
Basis Path Testing
In small groups Derive a data flow graph from the code (see
next slide). Define independence paths (basis set) Derive tests to drive each path in the basis set
51
Activity Code (Valid pins 10000-19999):
Procedure Validate_Pin (Valid_Pin, Return_Code)Valid_Pin = FALSEReturn_Code = GOODPin_Count = 0do until Valid_Pin = TRUE or Pin_Count > 2 or Return_Code = CANCELbegin
get Pin_Number (Pin_Number, Return_Code)if (Return_Code ≠ CANCEL)begin call Validate Pin_Number (Pin_Number, Valid_Pin)
if (Valid_Pin = FALSE) thenbegin
output “Invalid PIN, please re-enter PIN”Pin_Count = Pin_Count + 1
endend
endreturn (Valid_Pin, Return_Code)
52
Read Pressman Chapters 21-23 (Project Planning and Management) Mid-term quiz due (see course home page) Current event reports:
CastiglioneHillHoganSemler
For October 17
53
Extra slides
54
Change Control Process
Create InitialSections
Create/ModifyDraft
Review Draft(V&V)
Create Changes to Incorporate
Changes Needed In Document
DocumentApproved
Create Review Revise ReviewReview Approved
Time
...
Document in Production and Under Formal Change Control
Document in Production and Under Formal Change Control
Document Under Development and User Change Control
Document Under Development and User Change Control
55
Waterfall model
Systemrequirements
Softwarerequirements
Analysis
Program design
Coding
Testing
Operations
Source: Royce, W. "Managing the Development of Large Software Systems."
56
RUP Artifacts by Phase and Discipline
Discipline Inception Elaboration Construction TransitionBusiness Modeling
RequirementsVisionUse Cases (20-80%)ActorsSoftware Req SpecGlossary
Analysis & Design Software Arch Doc
Implementation
Build PlanBuildTest Results
Test
Test PlanTest ScriptTest DataTest Results
Test Strategy
DeploymentDeployment Plan Training Materials
Support MaterialsAcceptance Test ResultsChange Requests
Product
Executable ArchitectureUser Interface PrototypeUser Interface DesignUse Case RealizationDesign ModelDatabase Design
Business Architecture
57
RUP Artifacts by Phase and Discipline
Discipline Inception Elaboration Construction Transition
Configuration and Change Management
Project Management Risk ListRisk Mgmt PlanBusiness CaseQA PlanSoftware Dev Plan
Environment
Dev Case (Process)ToolsGuidelinesTemplatesSupport
CM PlanCM EnvironmentChange Requests
58
Technology
ProcessPeople
The focus of SE 425 is the process component of software engineering
Core Concepts
Technology
ProcessPeople
… for the delivery of technology-enabled business solutions
59
V-Model
IntegrationTest
Unit Test
Code
AcceptanceTest
SystemTest